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INTRODUCTION 

This sixth, and final, report of the Second National Study of 
PACE will feature special reports by 17 special consultants. Most 
of them have been with the study for two years. In the course of its 
work, members of the two study teams have visited close to 300 
PACE projects;, studied over 1,000 proposals, attended several 
meetings on ESEA Title III, and read numerous materials. Every 
team member took the assignment seriously, and each contributed 
time and interest considerably beyond what an already over committed 
schedule would allow. 

In this volume, each consultant views the future of PACE from 
the vantage point of his own speciality. Credit or criticism rests 
with each author. 

I would like to exercise a point of personal privilege and ack- 
nowledge my sincere appreciation to my wife, Peggy, who has under- 
stood the demands that PACE makes on anyone connected with it. 

This study is sponsored by an ESEA Title III grant, to the Cen- 
ter for Effecting Educational Change, Fairfax County, Virginia, which 
subcontracted the assignment to the University of Kentucky’s Research 
Foundation. 

Richard I. Miller 
Director of Study 
November 29, 1968 
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SOME INITIAL REMARKS 



^ / 

by Richard I. Miller*” 



On September 23, 1965. American education turned a historic 
corner when President Johnson signed the authorization bill for The 
Elementary and Secondary Act. While federal support of education has 
been evident at least from the Northwest Ordinance of 1787 and the 
Morrill Act of 1865, the 1965 authorization marked a degree of finan- 

cial support of unprecedented proportions. 

The most imaginative and exciting aspect of the ESEA package 
has been Title lU, known as PACE. Perhaps it would be useful at 
this point to review very briefly the uniqueness of this Title ^ con- 
ducted during the first two year^ . Seven points are given in the re- 
port of the first national study: 

1. Title III is unique in its broad mandate. 

2. It is 100 percent money to local agencies — real money 
for the first time. 

-^Director, Program on Educational Change, University of 
Kentucky; and Acting Chairman, Department of Social and Philosophi- 
cal Studies in Education. (And Director of the First and Second Na- 
tional Studies of ESEA Title III.) 
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3. PACE has a built-in requirement for community participa- 
tion. 

4. Title III establishes 50 state contests as well as one na- 
tional one since approval is competitive. 

5. It emphasizes innovativeness and creativity in its projects. 

6. The extent of Congressional interest in the program is 
unique. 

7. The federal- state relationship in unique. 

John W. Gardner allegedly said that "great ventures start with 
a vision and end with a power structure. " 

Is this true with PACE? 

How far, if at all, has this innovative and creative title been 
afflicted with hardening of the categories? 

The following 17 reports all approach this problem, and others 
related to the future of PACE. This form of evaluation — expert opin-~. 
ion — remains a reliable and important way of passing judgment. In 
an age of whirling computers, parametric measurements, and scienti- 
fic analysis, it is comforting to find that the voice of experience and 
honest inquiry still has a place in the scheme of things. 




CURRICULUM DEVELOPMENT 



^ / 

by William M. Alexander” 

This statement is based on an examination of two groups of 
Title 111 proposals, the first during 1966-67, and a second group during 
1967-68, and on site visits to a number of Title 111 projects. The state- 
ment also reflects the author's participation in various sessions of the 
PACE national study team and his general experience in curriculum 
development. 

New and Important Developments in Curriculum 

At least three major emphases in present curriculum develop- 
ment should be reflected in the innovative and creative educational pro- 
jects supported by Title 111: (1) the reorganization of instructional prO' 
grams in the major subject fields; (2) the development of packaged in- 
structional materials by various national groups, and (3) a pervasive 
emphasis on the individual learner through many instructional and or- 
ganizational practices. 

—'Director, Institute for Curriculum Improvement, College of 
Education, University of Florida. 
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The reorganization of subject matter through the "new" cur- 
riculum plans is a phenomenon of the last decade, spearheaded by the 
many national curriculum projects. New structures and sequences of 
subject matter have been developed in science, mathematics and foreign 

languages, and increasingly evident now in English, social studies and 

the arts. The so-called "reform movement" has touched in varying de- 
grees each area of the school curriculum. Already some of the new 
programs are becoming the old ones, being superseded in turn.by 
still newer ones. Continuing curriculum development should be ex- 
pected to maintain a continuing inquiry concerning the efficacy of new 
programs and ways and means for maintaining constantly updated 
curriculum plans. Furthermore, along with the plethora of new defini- 
tions of scope and sequence in the subject fields, there is an increasing 
concern for a proper fitting of the parts into adequate total curriculum 

designs. 

Related to the development of the new subject matter is the 
packaging of new programs into combinations of textbooks, program- 
med instructional materials, recorded and audio-visual materials, and 
various types of accompanying hardware. In some cases, aided and 
abetted by the national curriculum projects, industrial combines as 
well as other production and distribution agencies are today developing 
comprehensive instructional packages, kits and other combinations of 
projects. These developments should be accompanied by extensive 
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efforts in local school systems to study and try out various combina- 
uons in order to discover the right "packages" for particular learners 

or groups of learners. 

Current developments in curriculum and instruction direct a 
special focus on the program of the individual learner. Although con- 
cern for the disadvantaged learner has become paramount, comprehen- 
sive programs in local school systems are attempting ways and means 
of reaching each individual, with recognition of his particular capacities 
and potentialities, his learning processes, and evenhis idiosyncracies. 
The widespread interest in new organization of the school ladder, in 
individualized progress plans, and in independent study and other ar- 
rangements for individual learning activity, should be reflected in pro- 
grams of educational innovation and development in local school systems. 

PACE and the New Developments 
The Title III proposals thus far reviewed by this writer have 
not, in general, been focused on innovation in particular subject fields, 
since the national study team has included specialists in some subject 
fields. Several proposals advocating multi- service centers have pro- 
vided for assistance to schools in demonstrating and evaluating new in- 
structional programs and materials. One substantial effort in curricu- 
lum development, the Hawaii Curriculum Center, is making a unified 
systems approach to designing, developing, demonstrating and 
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evalaaung new curriculum programs and materials. This approach 
was especially evident in English, foreign languages and the fine arts. 



when visited by this reviewer. Except for this Center, no adequate, 
comprehensive approach to curriculum development is known to be 

operating through Title III support. 

The situation is similar with respect to the tryout and systema- 
tic evaluation of instructional materials packages. Most of the mul- 
tiple-service centers are assisting the local districts in identifying and 

using newly-developed instructional packages. Again, only the Hawaii 

Curriculum Center is known to be developing and evaluating packages 
comprehensively with the primary support of Title UI. 



The proposals reviewed and the sites reflect much more clearly 
an emphasis on identifying and utilizing a variety of arrangements for 
direct focus on individual learners. Note the titles of certain projects 



reviewed: 



"Provisions 



for Restructure in Independent Study Models" 



"Know and Care Center" 
Developmental Placement Program: 



tl 



"Model School Project to Adapt Exemplary Individualized 
Techniques to Limited Facilities of the Rural School 
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Motivation Through Enrichment 



"A Coordinated Attack on Underachievement Through 
Special Services" 



tl 



Center for Individualized Learning" 
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Also, five proposals were reviewed which aimed to emphasize indivi- 
dualization in new, middle school organizations, and several of the 
multiple- service center proposals emphasized the support of services 
and programs for individual pupils. In addition, many in-service edu- 
cation programs for teachers were directed at increasing the quantity 
and quality of individualized instruction arrangements. 

Major Obstacles to Better Curriculum Development Through PA^ 
Two approaches to better curriculum development are seen as 
appropriate for PACE support. The first, a widespread search for, 
and experimentation with, curriculum and instructional innovations is 
being relatively well accomplished through planning and operational 
projects in many school districts. The major obstacles to greater 
success with current expenditures would seem to be a lack of compe- 
tent leadership in some school districts (frequently those most lacking 
in innovation), and the lack of interest, all too often evident in the 
same districts, in fundamental educational improvement. The situa- 
tion is complex: better leadership would produce greater interest, but 
the present disinterest discourages the employment of leaders from 
outside, as well as the initiative of potential leaders inside the school 

district. 

The second approach, the comprehensive development of cur- 
riculum programs and instructional materials, is somewhat stymied 
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by the obstacles of costs and the contrary philosophies of curriculum 
development. As to costs, the problem is one of relative emphasis 
on large and small Title III grants. As the states take over the ad- 
ministration of Title III funds, each will face the issue of whether 
some substantial proportion of its Title III money should be put into 
one or more large operations. This reviewer hopes that several 
states will devote large Title III allocations to the maintenance of 
continuing centers for curriculum development and evaluation. 

The establishment of such centers indicates a reconciliation of 



opposing points of view as to the roles of national, state,, and local ed- 
cational agencies in curriculum development. Proponents of more at- 
tention to either national or local activity may oppose concentration of 
effort at the state level. Here, it is argued that under our Constitu- 
tion, and in view of the dangers of national curriculum development 
and the inadequacies of local programs, greater leadership by well- 
planned, adequately funded state centers is desirable. 



Future Directions for PACE 

The writer's hopes for the future of PACE may be inferred 
from the preceding sections. Specifically, three priorities are sug- 
gested for the allocation of Title III funds during the foreseeable 
future: 

1. Continued support of these local school district projects 
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which effectively search out, try out, evaluate and utilize promising 
innovative practices in curriculum and instruction. 

2. Support of plans for identifying, training and employing 
competent curriculum leaders in school districts lacking these per- 
ha.ps through cooperative arrangements for EPDA and other programs 
of internships in Title III projects — for training curriculum develop- 
ment and evaluation specialists, and for the creation of resource 
panels of curriculum specialists to serve those districts lacking such 

services. 

3. The establishment in several states of curriculum develop 
ment and evaluation centers organized around problems of statewide 
importance. Such centers could ultimately constitute a great network 
of federally- and state- supported, state- controlled centers to aid local 
school districts, first in their individual states, and later, perhaps 
elsewhere, as their products become available and prove useful. 
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THE CULTURAL ARTS 

^ / 

by Elliot W. Eisner^ 

During the past five-year period, a number of new and impor- 
tant developments have emerged in the cultural arts field. These 
I developments will be identified in this report and can be used, in part, 

I 

to appraise the projects in this area funded by Title III. 

Perhaps the most significant development in the recent past has 
been a broadened conception of curriculum content and curriculum 
goals in cultural arts education. In the past, the dominant view of ap- 
propriate curriculum content in the cultural arts was content which en- 
gaged students in productive activities. In the visual arts, children 
were engaged almost exclusively in tasks aimed at the production of 
art forms. In the area of music, children sang or played musical in- 
struments. While these forms of activity are important, they are not 
now considered to be exclusive by the ones to be provided for in arts 
programs for elementary and secondary schools. Joined with the 



■—^Associate Professor of Education ;and Art, Stanford University. 
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productive aspect of the cultural arts curriculum are two other com- 
ponents: the critical and the historical. The critical aspect of the arts 

curriculum attempts to develop competencies which will enable the stu- 
dent to respond in an aesthetically relevant way to art and musical 
forms produced by others. This aspect of the curriculum attempts to 
help the student to appraise critically what he sees, and to provide rea- 
sons for the judgment he makes about art forms. In short, the critical 
aspect of the curriculum develops a sensitive eye and ear, and a criti- 
cal mind that converts mere aesthetic preference to aesthetic judgment. 

The historical aspect of the curriculum in the cultural arts has 
been conceptualized because of the recognition that the cultural arts 
are products of culture, and that they can best be understood and ap- 
preciated when viewed or heard in the context of the culture in which 
they were developed. This is not to imply that a child can put himself 
into the Globe Theater to hear and see "The Merchant of Venice. '' 
Rather, it is to suggest that by understanding the Elizabethan era one 
can gain a richer view of Shakespeare's works. 

These aspects of the cultural arts curriculum have begun to 
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take on increased importance in the cultural arts fields. The older con 
cern with the development of creativity, emotional health and self-ex- 
pression have, to some extent, given way to goals in the arts which 
are more uniquely artistic in character. 



The concern for the critical and historical aspects of the arts 
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curriculum has been accompanied by the general idea of aesthetic edu- 
cation. While a common conception of aesthetic education has yet to be 
developed, there is general agreement that one of its major aims is the 
development of the sensibilities so as to enhance the life of feeling. In 
addition, those writing in the area of aesthetic education have placed 
an especially high value upon the development of powers of criticism. 
Aesthetic education seeks to develop the students' abilities to talk in- 
telligently about the arts, using the tools of formal aesthetics as well 
as art criticism. What makes aesthetic education different from a 
productive, critical and historical examination of single art forms is 
its attention to the arts as interrelated forms of human expression. 
Learning how to relate the artS; and to package them appropriately 
within the confines of a course is one major concern of many of those 
who work in the field of aesthetic education. Whether or not their ef- 
forts succeed remains to be seen; nevertheless, those writing about 
aesthetic education and those developing programs in aesthetic educa- 
tion have provided intellectual vigor and rigor to the cultural arts 
field. 

Directly related to these developments in the cultural arts is 
the realization that new forms of teacher education are necessary if 
the conceptions underlying new programs are to be successfully imple- 
mented. While in previous years art educators could talk about pre- 
paring artist- teacher s, this conception of the teaching of art is now 







- 14 - 



considered too limited. Teachers of the arts will need to be critic- 
teachers or historian-teachers as well, or at least specialists in these 
areas will have to be trained to work in the critical and historical do- 
mains of the arts curriculum. In short, with an expanding conception 
of curriculum content and curriculum objectives has come an expanded 
conception of the skills needed in the teaching of the arts. 

Two other developments worth mentioning have emerged in the 
cultural arts scene. First, the need for new instructional media has 
been recognized, and second, the need for effective evaluation proce- 
dures has become imperative. 

With the development of new instructional media in the sciences 
and social sciences, professionals in the cultural arts have begun to 
recognize the importance of multi-media tools for facilitating artistic 
learning. The single concept film, the use of specially prepared slides 
and the development of transparencies and specially prepared repro- 
ductions are now in the process of development in various parts of the 
country. Regional laboratories such as CEMERL and CAREL, as well 
as such universities as Stanford and Case-Western Reserve, are be- 
ginning to develop new instructional media to support those teachers 
using new curricula in the cultural arts. These materials will serve 
to facilitate aesthetic learning by allowing the student to compare and 
contrast the qualities constituting visual and musical form. And if 
these devices prove as effective as their advance billing would indicate. 
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it will be possible for students to develop levels of critical awareness 
in the cultural arts that are generally believed to be reserved for the 
artistically gifted. 

The move toward the development of new e\raluation mechanisms 
has been slow in coming in the cultural arts, but it is finally on its way. 
The paucity of instruments in this area is due to a variety of factors: 
little interest in measurement among those in the field, reluctance on 
the part of test publishers to publish a test for a limited market, and a 
dearth of people with competencies in test construction in the field of 
the cultural arts. In addition, although school districts frequently ad- 
minister tests to evaluate programs in mathematics, social studies, 
science and reading, there are very few, if any, school districts which 
assess programs in the cultural arts. 

Motivated by the Office of Education, and by a growing cadre of 
individuals sophisticated in both the arts and educational evaluation, 
instruments are being developed to assess highly complex forms of af- 
fective learning. These instruments, though primitive in form at pres- 
ent, will, when refined, make it possible to secure objective, reliable 
and valid data in the programs employed by schools in the cultural arts. 

PACE Developments in the Cultural Arts 

The Title III projects I have seen, and the proposals I have 
read, cover an exceedingly wide range of sophistication. Those at the 
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upper end of the continuum reflect many of the significant developments 
in the field. Such programs are concerned with using new media and 
modes of instruction to effect artistic learning. Special exhibition ma- 
terial, teacher manuals and other instructional resources are used. 
Sophisticated programs also attempt to develop or acquire evaluation 
tools useful for appraising their effects. And when the project staff 
does not have the qualifications to design evaluation strategies, compe- 
tent consultants are called upon. 

At the other end of the continuum are projects in the cultural 
arts which lack the human resources to carry off the task successfully. 
Project directors not sophisticated in the field may not realize the 
very real needs they have for using funds effectively. What I have 
seen and read of Title III proposals indicates to me that the entire 
panorama can be found in those projects which have been awarded 
funds. Ironically, it is generally the case that where the need is the 
greatest, such as in rural or isolated areas of the country, the com- 
petencies of the staff have been weakest. 

Obstacles and Challenges 

Obstacles to the development of programs reflecting the most 
advanced thinking in the cultural arts field are multiple, but the most 
formidable obstacle, in my opinion, is the lack of competent person- 
nel. Where the project director was strong and his staff competent. 
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the project was usually vital, the esprit high, and one could sense a 
feeling of progress. Where the director and staff were weak, one often 
sensed confusion in direction and bickering among the staff. 

But a competent staff does not automatically ensure an effective 
program. Some of the obstacles to effective projects were wholly uni- 
que to the situation. In one project, for example, it was not possible 
to hire the type of person needed for the project, although the project 
director was skilled and perceptive. In another project, the staff was 
enthusiastic, but the students* exposure to the program was so brief as 

to be unlikely to yield significant learning. 

In addition to these obstacles, there is a tendency on the part 
of projects to have an "add on" quality to them. That is, they tend 
not to become institutionalized in the school's program, but tend rather 
to function as enrichment programs — supplementary, rather than cen- 
tral aspects of the total educational program. Such plans probably re- 
sult from a type of educational timidity that avoids requiring any radi- 
cal change in the school program to permit the accommodation of a 
promising program in the cultural arts. Yet, to the extent that 
these programs remain peripheral, to that extent are they likely to be 
dropped when the flow of federal funds ceases. Cultural arts programs 
need to become a part of the mainstream of the schools' overall cur- 
riculum, and students need long-term exposure to such programs if the 
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goals of those, who are at the cutting edge of the cultural arts field are 
to be attained. 

Among the obstacles to successful Title III programs in the cul- 
tural arts are the lack of competent personnel, personnel who know 
both the cultural arts and curriculum and evaluation; a peripheral qua- 
lity to the programs, that is, the development of programs with an en- 
richment or supplementary quality to them; and other obstacles unique 
to the context, and which are not generalizable. 

Looking Ahead 

There are no simple solutions to the problem of improving cul- 
tural arts programs in Title III. The problems faced by projects are 
diverse and complex. In addition, the conceptualization of the field of 
cultural arts is itself undergoing change, and people in it are feeling 
their way through a variety of schemes for rationalizing its structure. 

I believe, in general, that projects can be improved if they will 
attempt to provide sequential and continuous programs which go beyond 
brief exposure. To say that 60, 000 children have been served by a 
project is insufficient if their contact with the program has only been 
from twenty minutes to a half-hour. Too many programs try to reach 
too many children. It would be far better to demonstrate the effect of 
an intensive program than to provide a superficial program for larger 
populations of students. 
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The development of instructional materials in conjunction with 
the program holds great promise, in my opinion, for improving cul- 
tural arts projects. Materials once produced become a part of the 
teachers' instructional resources and can be used over and over again. 
They seem to anchor the project in the classroom and, in addition, 
they provide the teacher with the help needed in instruction. 

The use of professional artists on a short-term basis appears 
to be an effective way of motivating both students and teachers. Child- 
ren see an individual who was only an image before, and perhaps not a 
flattering one at that. Teachers, too, have an opportunity to observe 
technique and teaching method. The enthusiasm of interested artists 
appears indeed to "rub off. " 

Lest this final report give the impression that the cultural arts 
programs in Title III are feckless, let me here dissipate that impres- 
sion. There are, to be sure, projects not likely to make a significant 
impact. There are such projects in all fields. But, Title III has made 
it possible for school districts to secure programs which would never 
otherwise be offered to students. In doing this, it has made a contribu- 
tion to American education. The beginning of change, the opening 
wedge, has been initiated in many districts through the efforts and sup- 
port of Title III programs. Whether these small changes will continue, 
or whether the "crack in the plaster" will be returned to its previous 
rigid state, remains to be seen. But Title III projects have made a 
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beginning by demonstrating that programs can be done differently. To 
demonstrate this in American schools is no small achievement. 
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HANDICAPPED CHILDREN 

*/ 

by Lloyd M. Duninr" 

Title III of the 1965 ESEA (or more popularly, PACE — Projects 
to Advance Creativity in Education) was designed to stimulate innova- 
tion and exemplary projects in all of education, including special edu- 
cation. PACE has only been in operation for three years. In that 
time, it is estimated that approximately 150 proposals have been re- 
ceived in the area of education for the handicapped. While those funded 
have resulted, no doubt, in new programs which would not otherwise 
have been initiated, the large bulk of these proposals could be charac- 
terized as being diverse in nature, rather traditional, and generally 
lacking in evaluation. This is not surprising since they were an un- 
solicited, uncoordinated group, submitted ad lib by a variety of local 

school systems. 

Now there have been two developments. First, Title III is being 



—^Director of the Institute on Mental Retardation and Intellec- 
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passed, in large measure, to the states for coordination through state 
plans. Second, 15 percent of the Title III funds have been earmarked 
for special education projects. Thus, it is an opportune time to pro- 
pose some promising directions for special educators for investing 
their quota of venture money allocated for innovation. 



Promising Directions 

The hazards of making projections are well documented in many 
areas of human endeavor. Man has been remarkably inept at seeing 
changes needed in the social order, and even more unwilling to imple- 
ment them. Traditionalism, conservatism, and an inability to change 
have characterized nearly all social institutions, especially that of 
education. There are a few signs that these same phenomena are be- 
coming widespread in special education for the handicapped as it be- 
comes more entrenched. Notwithstanding, there are even stronger 
indications that much of special education is not satisfied with its pres- 
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ent practices, and is not prepared to become a program based on tradi 
tional philosophy and expediency. If this latter reading is more cor- 
rect, then it would seem that innovations in the following five areas 
show the greatest promise for initiating an American Revolution in 



Special Education. 



1 . Boarding schools . 

In the United States, the standard practice has been for State 



















-23- 

Departmehts of Education to operate residential schools for the blind 
and for the deaf, and in certain cases, for dependent, emotionally dis- 
turbed, delinquent and mentally retarded children. However, in these 
latter cases, it is usually some other department of state government 
which assumes this responsibility, including such departments as 
health, mental health, corrections, and so forth. Through the years. 
State Department$ of Education have been very reticent about operating 
residential facilities, except in the areas of the deaf and blind. HoW" 
ever, the time has clearly come to devise and to study the efficiency 
of a wider variety of special-purpose boarding schools for the handi- 
capped. The need is particularly pressing in such areas as the socio- 
culturally deprived retardate, the emotionally disturbed and the mul- 
tiple handicapped. With regard to the multiple handicapped, there are 
a number of issues that need to be resolved. How specialized should 
these boarding schools be? Should there be a different one for the 
deafrblind, the blind- mentally retarded, the cerebral palsied-men- 
tally retarded, etc. ? Or should children with a wide range of learning 
problems be served in one facility? Of course, the size of the facility 
is at issue. It probably should not exceed 35 to 50 pupils. By broaden- 
ing the function of a unit somewhat, one can bring in a wider array of 
educational specialists to work with the child. Too, consideration 
should be given to providing day and boarding school services at these 
centers. In any event, when developing its plan under Title III of the 
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ESEA, each state should deliberate hard and long on the new need for 
small, special-purpose boarding schools for children with handicapping 

conditions. 

There are many conditions which make it increasingly difficult 
for many handicapped children to make adequate progress in the special 
day programs provided by local school systems. Increased family 
disintegration is upon us. Thus, the home often does not provide the 
warmth and security which so many children with severe learning 

problems need. Furthermore, physicians continue to save children 

who would otherwise have died in the past, children with residuals be- 
cause of high fever from rubella, etc. , accidents, or other catastro- 
phic events. Furthermore, with the population explosion and problems 
connected with birth control, it is obvious that, in the immediate fu- 
ture, we are going to have many children with more severe learning 
problems than those we have had in the past. It is time now, not only 
to begin to serve adequately the children who are already at hand and 
who need our help, but also to evolve and field test programs for those 
children who will be at our doors in the immediate future. For many 
of these present pupils and pupils -to-be, there is growing evidence 
that optimal education cannot be provided through day-school program 
which operates only a few hours a day, for about 185 days a year. 

What is needed is an around-the- calendar as well as an around-the- 
clock program for certain children. Intensive and continuous 
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instruction could be provided in a boarding school by having two sets 
of teachers; one to work during the regular school hours, and another 
group to work with children outside the regular school hours. With 
such a regimen, it should be possible for the pupils to make much 
more progress than has been the case in the past. Titlo lU should 
give us an opportunity to try out small, special purpose boarding 
schools, possibly by renting, renovating, and vising suitably large, 
old residences near colleges and universities with faculties which 
could provide expertise in program development and evaluation. 

2. Educational diagnosis and prescription developing centers. 

One of the most heartening developments in special education 
is reflected in the receipt of applications for funds under Title III to 
try out new procedures for replacing past diagnostic practices. 
Traditionally, special education diagnosis has been done by the psy- 
chologist, physician, social worker, speech and hearing specialist, 
and sometimes, educators. Generally, the goal has been to find out 
what is wrong with the child, and why. Thus, the result has been, 
understandably, to place a disability label on the child, based largely 
on the medical model. Many children with a common disability label 
have been homogeneously grouped under some special education ser- 
vice. This has created a self-fulfilling prophecy in that the effects of 
these disability labels have been devastating. First, they have not been 
badges of distinction and therefore have stigmatized the child. Second, 
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they have lowered the level of aspiration of the teachers in their in- 
struction. Third, they have not enhanced the self-esteem of the child- 
ren so labelled. There is a strong trend in the field to do away with 
both our traditional areas of disability and their disability labels for 
the more mildly handicapped. Here the common terms refer to the 
educable mentally retarded, the minimally brain injured, the mildly 
emotionally disturbed, and so on. Instead of looking to physicians and 
psychologists for educational diagnosis and recommendation of special 
education placement, a new pattern has emerged. Clearly, it makes 
sense for the profession providing the treatment to be responsible for 
its own diagnosis, using the ancillary help of other professions as 

needed. 

What is proposed is that sizeable school systems (or two or 
more small school systems in collaboration) establish Speci al .Ed^- 
ratinnal Diagnostic and Prescription nevelooment Centers. These 
centers would be staffed by an array of educational specialists with 
expertise in such areas as motor development, sensory- perceptual 
training, cognitive development, language development, remedial edu- 
cation, and so forth. The role of this team would be to take a child 
and work with him long enough to ascertain basic information on where 
he Is operating, to discover the channels by which he can learn best, 
to determine adequate mechanisms for his motivation, and (most im- 
portant) to devise, try-out, and establish a successful method of 
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instruction. Thus, children would need to attend this facility for at 
least a month, and hopefully, until a successful intervention program 
had been devised for him. Clearly, this does not necessarily do away 
with labels, but it certainly removes the need for using those 
ability labels which have done more harm than good. Instead, from 
time to time, a child might have a descriptive label which would des- 
cribe the program in which he is enrolled; for example, in language 
development versus sensory training. Not only would physicians, 
psychologists and social workers be brought in as consultants when 
needed, but children with apparent psychological, physical or social 
problems could be referred to the appropriate authorities. However, 
the clear emphasis would be on the responsibility of educators for their 
own diagnosis and for devising their own interventions. Hopefully, the 
risk capital provided under Title III would enable State Departments of 
Education (in collaboration with selected local school systems) to ex- 
periment with this new approach to educational diagnosis and interven- 
tion. Rather obviously, our traditional procedures have been ineffec- 
tive. 

3. Curriculum development . 

If special education is to move from an intuitive clinical basis 
to a science of instruction, much time, effort and talent must be de- 
voted to curriculum development. This will be difficult for those 
special educators who feel that all pupils are different and need unique 
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courses of study. Understandably, many are unable even to entertain 
the notion that standard sets of instructional materials (or sequential 
lessons) could be useful in teaching handicapped children. Hov/ever, it 
can be argued that well-programmed instructional programs do have a 
place in special education when viewed as providing the basic ingre- 
dients which need to be adapted and modified to suit a particular child 
with a particular set of needs and learning characteristics. In my 
view, essentially all Title III funds could be wisely spent on curricu- 



lum development. 

In terms of a modus operand! , it would probably be wise to set 



up a variety of teams of specialists across the nation to work on speci- 



fic areas. Tim 1 rhni- Curriculum Developme nj 



Centers is advocated. In terms of manpower, the teams would vary. 



depending on the area of emphasis. For example, in working up speci- 



alized programs in motor development, we would need teams of special 
as well as physical educators, and occupational and physical therapists. 



not to mention people in recreation, physical education, etc. In the 



areas of language development, speech pathologists, audiologists, speci- 



al educators, psycholinguistic specialists and other language develop- 



ment authorities would need to form the team. It would seem wise to 



release from classroom instruction some 10 to 15 percent of the most 



creative special educators for participation in these materials develop- 



ment ventures. The first step of these teams would be to conceive of 
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a conceptual model on which to base their programs. Teachers and 
theoreticians would need to collaborate. It would be wise, in many 
cases, for these centers to be established at. or near, colleges and 
universities. However, the other option would obviously be to keep 
them within the state and local school systems. Extensive and eiabor- 
ate sets of exercises would need to be devised in a number of basic 
areas. Such broad topics as the foliowing might provide a framework 
for organizational purposes: (1) motor development, (2) sensory 

and perceptual training. (3) cognitive and ianguage deveiopment in- 
cluding the academics, (4) speech training. (5) connative develop- 
ment. (6) social development, and (7) vocational training. After 
the various models have evolved, the task of the teams would be to 
create, field test, modify and test the efficacy of the various pro- 
grams. It is not recommended that the teams work along the lines of 
the traditional disability labels, but that they be grouped according to 
what they can contribute to a particular area of instruction. For ex- 
ample, in the area of auditory training, people in remedial reading, 
education of the deaf, audiology and education of the mentaily retarded 
might combine their talents to devise a program better than one devised 
by each group operating independently. With an adequate investment 
in curriculum development, perhaps in a decade or two we will no 
longer be nonplussed by the question: What is special about special 

education for the mildly handicapped? 
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4. Manpower for special education- 

As in all of education, special education has had a perennial 
shortage of master teachers. In the foreseeable future, this critical 
problem will not be alleviated. Thus, the need exists for innovations 
in utilizing master teachers most effectively. In this regard, the 
vertical team notion is presented for experimentation. Here a master 
teacher would be at the top of a pyramid, below which would be standard 
teachers, beginning teachers, apprentice teachers, student teachers, 
community volunteers, teachers' aides and perhaps others. Often, 
handicapped children need to be taught in a one-to-one tutoring rela- 



tionship. But there are, not enough master teachers to enable each 
handicapped child to have such a person as a tutor. However, the no- 
tion of a cadre of persons concentrating on a group of handicapped 
children might permit much more tutoring and intensive instruction 
than is presently the case. Experimentation with the vertical team no 
tion would appear to have merit, utilizing Title III funds. Certainly, 
the master coordinating teacher should have a large salary supple- 






ment for the extra work, responsibility and talents that would be re- 
quired for success in this field. A salary from 25 to 50 percent higher 
than that of the standard teacher would not seem inappropriate. This 

would, of course, need to be built into the plan. 

Generally, special class enrollments have been quite small, 
ranging from as low as five or six pupils, to a high of 15 or 16. With 
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a vertical team available to a group of children, a master teacher 
could handle as many as perhaps 30 or more children, probably more 
effectively than she is now able to handle 12, on her own and without 
extra help. Experimentation with the type of manpower and size of the 
group could help us now. Even more valuable would be the informa- 
tion gleaned from such experimentation as we attempt to extend ser- 
vices to the increased numbers of handicapped children who will be at 
the school doors in the foreseeable future. 

5. Early childhood education . 

Some progress has been made by special educators in beginning 
special education in the pre-school years. Perhaps this is most true 
in the area of education for the deaf. Here, it has been rather clearly 
demonstrated that special education needs to begin as early as the age 
of three to assure deaf children of a reasonable chance of developing ef 
fective communication skills. Small beginnings have also been made 
at providing early childhood education for children with certain other 
learning disabilities. However, as yet, most retarded, cerebral 
palsied, deaf, blind and other types of children do not begin their for- 
mal education until they reach the age of six. As Benjamin Bloom of 
the University of Chicago pointed out in his book entitled Stability; 
Change in Human Characteristics , almost half of all growth in human 
intelligence takes place before the age of four years, another 30 per- 
cent between the ages of four and eight, and the remaining 20 percent 
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between eight and seventeen. Thus, about 50 percent of a child's in- 
telligence takes place before the age of six, and 80 percent is com- 
pleted by the time he is eight years of age. There is growing evidence 
that other sets of behaviors are also largely acquired at a very early 
age. These data indicate the need for early education. Clearly, the 
years before six are especially crucial for children with potential 
learning disabilities. 

In the past, much of special education in local school systems 
has been a service for children who originally enrolled in the regular 
grades at the age of six, but who encountered learning difficulties in 
the primary grades. Regular teachers have then referred the children 
to special educators to remediate, or live with, the learning difficul- 
ties of the pupils with whom they have been unsuccessful. Generally 
speaking, special educators have not been much more effective in help- 
ing these children than have the regular classroom teachers. Thus, 
there has been growing disaffection with this referral and special 
education process. Instead, a new pattern of special education seems 
to be emerging. One is developmental in nature; the other, supportive. 
In developmental special education programs, special educators assume 
responsibility for the total education of severely handicapped children 
from an early age. In supportive special education, general education 
would continue to have central responsibility for the vast majority of 
the children with mild learning problems, with special educators 
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serving as resource teachers in devising effective prescriptions and 
tutoring such pupils. In these developmental programs, it will be ex- 
tremely important for special education to experiment with innovative 
programs in very early childhood education. Perhaps the notion of 
critical period has special relevance here. For example, perhaps the 
first two years will be especially critical for children encountering 
problems in the area of motor development. For those children en- 
countering their greatest difficulties with sensory and perceptual de- 
velopment, the period from one to two years would seem to be especi- 
ally critical. In the area of language development, between the ages of 
two and three is obviously the critical period. And so it goes with 
written language probably not becoming a critical matter until the age 
of six or more. However, the initial steps of learning toward academic 
instruction would need to be well established (namely, in motor de- 
velopment, perceptual development and language development) by the 
time the child reaches six. 

Title III funds could provide needed resources for experimenta- 
tion with very early childhood education programs for children with 
j_0aming and potential learning difficulties. A special challenge, in 
terms of Bloom's data, is that every effort be made to stimulate cogni- 
tive development in children with intellectual deficits, long before the 
first six years. Thus, it may be possible to initiate a program for 
prevention of the need for special education during the usual school 
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years, especially for children who come from backgrounds of socio- 
economic deprivation. Hopefully, children of slow- learning parents, 
of parents who have older siblings who are retarded, etc. , could bene- 
fit especially from cognitive training in the early childhood years. In 
fact, with such programs, it might be possible to advance cognitive 
development to the point where these children could attend a regular 
school program where compensatory education services are increaing- 
ly available. Comparative studies in early childhood education would 
need to be conducted to contrast, for example, the Montessori ap- 
proach with the Bereiter and Englemann procedures. Clearly, much 
innovation and experimentation will need to take place before the most 
effective procedures are determined. 

Concluding Comments 

This, then, is my blueprint of priorities for utilizing special 
education’s share of the Title III funds. Experimentation with (1) 
boarding schools, (2) educational diagnosis, (3) curriculum develop- 
ment, (4) vertical professional team, and (5) very early childhood 
education would, in my view, be the areas of greatest promise. Hope- 
fully, the PACE program in special education will move us, during the 
next decade, from teaching by intuition toward a science of instruc- 
tion. 
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PUPIL PERSONNEL SERVICES 

* / 

by Arthur A. Hitchcoclc" 

Four developments in pupil personnel practices are delineated. 
They are new in the sense that they now appear to be accepted in some 
degree but they have not become so dominant so as to rule the work of 
pupil personnel specialists. 

1. Theoretical concepts of the nature of man as determinants of pro- 
fessional worker roles 

For many years, the work of psychologists, counselors and 
social workers, has been viewed as that of helping students to be what 
they may become (and this is sound), but the processes have not been 
effective. Now, a theoretical concept is emerging — still tentatively, 
but viable — a concept new in the sense of its application to specialists 
in pupil personnel services. The subject matter of these services is 
man. We are concerned about his humanization in a very difficult 
world in which only the more-human human is likely to be able to 

^ / 

— Professor of Education, State University of New York at 
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eate a viable self. Pupil personnel specialties will shape their roles 



in 



accordance with the nature of man, man as an organism of needs. 



and the particular specialty involved. This theoretical concept forces 
the education of specialists into a thorough amalgam of the disciplines 
concerned in understanding man. Man as a cognitive-affective being 



is what guidance and counseling (and other services) are all about. 
This concept is not widespread; it is a new development, but it is also 



a direction ahead. 



2. Role identity and preparation for the role 

The one year overview of counselor preparation at the graduate 



level appears definitely to be out. Although the majority of counselors 
are still prepared at the one-year level, nevertheless the counselor as 



a technician, with one year of background, is inconsistent with the de- 



velopment and direction stated above. The change to more adequate 
preparation is being forced by persons of creative leadership in coun- 
seling, by new counselors who report that they do not have the back- 
ground for the difficult role of a counselor today, and by state depart- 
ments of education, the first of which — New York State — is moving to 



two years of preparation for counselor certification. The counselor as 



an identifiable person with a unique role cannot be supported by a 



technician's preparation. 

3 , The practice of counseling as ^ one - to - one relationship ^ changing 
Counselors in practice, particularly with educationally and 
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socially deprived students, are finding that exclusive use of the one-to- 
one traditional counseling relationship is neither applicable nor effi- 
cient. Proportionately, individual counseling is diminishing and group 
approaches with students are increasing. The independent counseling 
practice with students is also changing as counselors become more in- 
volved with parents and teachers to influence their effect upon students 
and student environments. With parents, group approaches have been 
found to be especially effective. The acceptance by counselors of com- 
puters is also influencing the counselor relationship. Computer- 
assisted counseling has been accepted, even though such systems really 
are not yet in practice. The help that can be derived from a computer 
system emphasizes the changing role of the counselor. The counselor 
now can have the help of a computer, thereby freeing himself for higher 

level actions. 

4. The pupil personnel concept is widely accepted in practic e 

Throughout the Title III study, the term, "pupil personnel," has 
been used. In fact, however, up to a year ago, pupil personnel ser- 
vices were accepted more as theory than fact. The leadership of New 
York State in this area was alone for many years. Now, however, not 
only is the idea widely accepted, but also the practice. This develop- 
ment is now recorded as fact because the concept of pupil personnel 
services in practice is important for the new directions ahead. New 
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dircctions csin become established under practices of the pupil per 
sonnel idea. 

These developments are set forth as specific, significant 
changes that can be pointed to. They are explained in a setting of con- 
siderable unrest in pupil personnel services. Counselors, especially 
those with only a few years of experience, are expressing their dis- 
satisfaction with a continuation of life as usual when life conditions 
have changed so drastically. Counselors should emerge into new roles. 

Today, counseling is generally accepted as part of the elemen- 
tary school program. Counseling is found broadly in the U. S. Depart- 
ment of Labor and Office of Economic Opportunity programs. Atten- 
tion now being given to the American Indian and to the Spanish- Ameri- 
can inevitably carries a counseling component. With these, and other 
extensions of counseling, practitioners are moving. But there are 
many problems and obstacles. 

PACE Programs and the Cutting Edge 
The cutting edge is the line which connects new developments 
and new directions. The first year of Title III gave schools the op- 
portunity to come up to date, to fill some of the holes in the education- 
al programs, to make repairs in the system. These measures had to 
be taken before schools could move on to change, to innovation, and to 
revolution. Now, the second year, this move has been made. Schools 
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with Title III projects in pupil personnel services have wrought changes 
which preclude any return to the business of old. During this year, 
PACE projects have been on the cutting edge — on that line which con- 
nects new developments and new directions. 

These are only examples to illustrate the point: 

1. The training and use of guidance aides and other para- 
professionals, after some five years of endea.'oring to persuade pupil 
personnel workers to employ such persons for work they could do more 
effectively than the professionals. 

2. The use of the computer for more than class scheduling 
and routine record keeping. PACE has funded projects to move the use 
of the computer into direct support for counseling, and for educational 
and vocational information systems. Such use is not widespread, but 
PACE has made it possible to cut the edge. 

3. Guidance and counseling as a rehabilitative agent. Inno- 
vative PACE projects are found with delinquent children, pre-delin- 
quents, and the emotionally ineffective. These are programs in which 
the ideas of rehabilitation rather than welfare have been present for 
several years, but it has been impossible to mount the program. 

PACE again provided the edge to experiment with the cutting. 

4. Inner city vocational incentive programs. Although pupil 
personnel services have not cut through to perform their roles in inner 



















city education generaJly, nevertheless, PACE- supported projects have 
illustrated what can be done. 

5. An honest and functional union of curriculum, instruction, 
and counseling. After many years of wishful talk about this three-way 
marriage, PACE moved some projects out on the edge and established 
the marriage as sound, and, in the projects visited, developed prac- 
tices that promised more valuable and exciting educational experiences 

for students. 

These are but five examples. While more could be added, 
these are adequate to support the contention that PACE has advanced 
onto the cutting edge. In the first section, several new developments 
were mentioned. These are large-scale. Each has been advanced by 
specific PACE projects. The sampling of PACE programs cited above 
give specifics which support the large-scale developments. 

Title III has given the pupil personnel community an opportunity 
to show how it can affect student life. PACE has, in fact, even forced 
student services to move in more significant directions. 



Obstacles to Achievement 

Obstacles to achievement are a mixture of PACE itself and the 
field of pupil personnel services. The obstacles explained below have 
been derived from studies of PACE projects and from studies of the 
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field of pupil personnel services. They constitute a mixture of the two 
elements. 

1. Theoretical aspects of pupil personnel services 

There is a tendency to think of education in terms of socializa- 
tion — and therefore efficiency — rather than humanization. This is not 
to denigrate the significance of achievement in American society. It 
is rather, however, to place the emphasis upon the ability of a person 
not only to achieve, but also to have a totally more productive kind of 
life, if his educational experiences have encouraged him to develop as 
a human being. Pupil personnel services too frequently are built upon 
the theory of services as such, rather than upon human beings as such. 
Very few school systems examine and continuously study the needs of 
students for human development, then building their services, curricu- 
lum and instruction upon those needs. This nation has reached the 
point at which pupil personnel services must meet the human needs of 
society if that society is going to emerge as greater and more livable. 

2. Professional obstacles 

As one exatrines graduate programs of preparation for coun- 
selors, one is appalled at their sterility. A basic combination of 
courses and areas of courses (such as occupational information and 
counseling techniques) has changed very little during the years. This 
is a generalization, and does not apply to all programs. There are a 
few that are moving more decisively into the concerns of society for 
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the development of human life. But, by and large, graduate programs 
in counselor preparation, to say nothing of psychology and social work, 
have not moved with the times. There are many reasons for this, but 
they cannot be the concern of this paper. The situation is exacerbated 
by the proliferation of non-quality programs of preparation. For ex- 
ample, in New York State, there are 37 institutions of higher education 
which purport to prepare counselors. There are not even that many 
institutions of high-level graduate education in New York State, much 
less that many which deal with the specific field of counselor prepara- 
tion. It has become very popular to prepare counselors. One obstacle 
to PACE development which will be considered next reflects upon this 
personnel situation. It is helped neither by sterile graduate programs, 
nor by programs so thin that they cannot possibly produce competent 
counselors, psychologists and social workers. 

A further obstacle in the professional area is the technician 
concept so strong in the less competent schools of preparation. These 
schools boast that they are "practical, " but in that practicality, they 
prepare people to perform at a minimum level of competence in the 
schools, and then call that level the "optimum." 

Also, one must recognize realistically that the professional 
societies have the difficult role of projecting higher standards for the 
field they represent, and at the same time, protecting the members. 
This represents a very fine line between progress and advancement on 
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the one hand, and the stultifying effects of the existing power structure 
on the other. This is simply a reality of life. At times, the profes- 
sional societies advance on bold fronts because they are in a condition 
to do so in the society, but at other times, they become obstacles to 
progress because they are so protective of their members and of the 
standards of preparation and performance they have promulgated. 

3. The personnel situation 

It was observed early in the National Title III Study that one of 
the national problems which could very well harm Title III projects was 
the lack of competent personnel. Fortunately, the projects observed 
have been able to attract competent (and often superior) personnel be- 
cause the projects are both interesting and moving. Yet one would be 
quite blind not to see that the personnel problem is the kind of obstacle 
that can prevent PACE from having as wide or as deep an impact on 
society at large as it might have. However, tied with this condition is 
the fact that more efficient use could be made of personnel. Many 
PACE projects have themselves illustrated this far better than the 
usual routine of pupil personnel services. In fact, this is one of the 
significant contributions PACE is making in pupil personnel services. 
The obstacle remains, and even partial solutions to the personnel 
shortage must await a more widespread emulation of projects already 
in effect in PACE. 
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4. Practices often are immune to change 

In many states, but not all, the certification requirements tend 
to present two obstacles to movement. One, they encourage the think- 
ing that minimum standards for certification are actually optimal 
standards; two, the certification practices put ceilings over, and 
walls around, the allowable activities of persons in schools. In pro- 
fessional fields, it appears that licensure by the profession, even 
though administered by the state, will eventually be the only suitable 
way out of the present condition. Under such licensing, we would have 
the fullest assurance possible that a person coming into the professional 
field was qualified to perform. Thus, he would be licensed to perform 
by the professional group, but not certified by the mechanism of state 
departments of education. 

Another obstacle found in practice is the feeling frequently ex- 
pressed by persons in the field that each year they are turning out a 
larger and larger number of persons who are going on to college. In 
other words, a situation exists in which there is a devotion to the same 
things that have been done for years. This attitude of meeting the par- 
tial needs of a part of the school population tends to prevent pupil per- 
sonnel specialists from moving into a future characterized of necessity 
by meeting the needs of a highly diverse population. This comment 
does not detract from the "new breed" who are seeking to move in 
more human- like directions, but it does say that the institution of 
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counseling, psychology and social work is in the control of persons 
who tend to obstruct change. 

5. PACE tends to fragment instead of expand 

In reviewing a large number of PACE proposals, and in ex- 
amining many of the projects in action, it is obvious that PACE has 
selected outstanding projects and has then encouraged them to move. 
Yet, despite all the remarkably fine models PACE has thus far sup- 
ported, one wonders if there might not be a greater impact if a balance 
could be established between the few, very large-scale programs with 
potentially national influence (programs which PACE has taken the 
leadership in encouraging) and those smaller, but very unusual, pro- 
jects initiated by highly imaginative local people. The obstacles sug- 
gested here is that models which are localized only tend to detract 
from the potential impact of PACE. There should be a balance. 






At various times in the course of the National Title III Study, 



obstacles have been observed and reported. The obstacles reviewed 
here are the deeper, underlying obstacles that affect both the field it 
self and the role of PACE. They are interrelated. But, despite any 
obstacles, it is obvious that PACE has had a profound effect upon 
movement within pupil personnel services. 



Future Directions 

The future directions of PACE depend, in one sense, upon the 
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future directions of pupil personnel services, but in a stronger sense, 
pupil personnel services may well depend upon the directions that can 
arise through the innovative element of PACE. 

First, a look at the directions of pupil personnel services which 
seem to be most significant, if they are to respond to the role in society 
that national policy, expressed through national legislation, indicates: 

1. Humanization of the experience of education 

This means that the counselors, in particular, must direct their 
efforts on two fronts: first, immediately with students in creating an 
individual growth of human worth toward a higher level of (most valued) 
human attainment; second, directly with teachers in creating an am- 
bience of humanness. The point is well explained in the research re- 
ported by Dr. Pauline Sears: ’’Rewarding children by personal interest 
and praise for personality traits, in place of rewarding only by work 
evaluation, produces children who perform and score higher in crea- 
tivity dimensions. . It is the responsibility of those in pupil person- 
nel services to effect a blend of the cognitive and affective in education- 
al experiences, directly with students, and indirectly through teachers. 

This direction of pupil personnel services may seem to conflict 
with the vocational guidance emphasis of today. Not so. But not so, 
only if career and educational planning and development are viewed 
as part of the total life style creation so vital in today's society. This 
direction, however, is in conflict with the guidance functionnaire view 













which S66S guida.nc6 as a series of mechanistic manipulations, 

2. Systems approach to pupil personnel services 

A limited guidance program in a small community should en- 
gage in an analysis of any existing program of guidance. For any city 

a/ 

with a population of 75, 000 or more,” an analysis of personnel ser- 
vices for students should be considered mandatory, with the under- 
standing that the services then are viewed as a system. The systems 
approach encourages a more efficient use of these services. 

The systems approach also, almost inevitably, brings about an 
evaluation system. This has become one of the greatest needs in pro- 
grams of pupil personnel services. Very few school systems, if any, 
honestly know what is happening to their products (the students) as a 
result of the processes of the pupil personnel system or program. 
Neither do they know the relative return for expenditures in the various 
specialties and sub- programs within the total system. 

3. Value impact on disadvantaged populations 

American society will never be able to turn its back on various 
disadvantaged populations. Results may not be very encouraging, but 
the social commitment is there, as is the devotion to the idea that 



-^According to the 1960 census, there were 196 cities of 75, 000 
or more in the United States. The number of such cities would be much 
larger today. In a city of 75, 000 population, there would be approxi- 
mately 14, 000 students. This is a large enough school population to 
merit a rather substantial array of pupil personnel specialties. 
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education is essential for creating the changes that such populations 
desperately need. Part of the current discouragement rests in the 
failure of pupil personnel services to penetrate beyond the level of 
superficial needs. This failure means that the totality of education is 
unbalanced. Yet, counseling is one element which society has decreed, 
through legislation, must be available to, and have an impact on, dis- 
advantaged children. One of the prime directions of pupil personnel 
services in the future must be toward a value impact on disadvantaged 
children. 

Now, a look at directions for PACE in pupil personnel services. 
The directions for the services, explained above, are deep, global, and 
far-reaching. PACE has already affected the new developments. Now, 
we must look ahead and see the influence that PACE can have. How 
can PACE affect these directions and thereby shape pupil personnel 
services as a viable force in American education? 

1. Deliberately structure projects , on a large scale , in each direction 

Many persons of influence in the structure of American educa- 
tion have been seeking larger, impact- laden projects. That is the 
need in this area. PACE could develop a multi- school district pro- 
ject to study, discover and show what changes can be effected in child- 
ren by the humanization of education. It could focus on the pupil per- 
sonnel services, changing their directions quite drastically, and at 
the same time affect curriculum, instruction, and administration. 
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The change in student development toward more value-directed lives 

in our society might be quite startling. 

The same could be said of the two other directions. The move 
for a systems approach to pupil personnel services could revolutionize 
these services and their impact on students. But, this is a large-scale 
project which would have to be staged in more than one school district. 
In the direction of an impact on disadvantaged populations, there should 
be a total- immersion project in several different population groups. 
There have been many projects on a relatively small scale, but now 
there should be an attempt to test new practices, some of which are 
suggested in this paper, on a much grander scale. 

A basic reason for this larger approach is the necessity for 
movement within a fairly short time. Student needs for human de- 
velopment will not wait for years, and PACE is the agent that can 
create these basic changes. Once they are present on a large enough 
scale to be visible and sound enough for transfer into other school 
systems, some means for communication must be evolved. The next 
two measures should help in this process. 

2 . A milieu for the generation of new and re - creative directions_ 

PACE could launch a national seminar. Many national confer- 
ences suffer slippage because the impact reaches so few of the people 
who make a difference. PACE could conduct a national seminar on 
humanization in education, in many, many locales for a period of one 
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year, with an appropriate follow-up in literature, directly with, and 
to, individuals who do make a difference on the working front, the 
local area. Other national seminars in the realms of the other direc- 
tions might well be mounted. Certainly, a continuous national semi- 
nar on counseling with the disadvantaged could have a tremendous ef- 
fect. 

3 . Evolve _a continuing re - generation agent 

PACE could move on two fronts to create a re-generation a- 
gent: One, support an institute, or similar agency, to serve as the 
continuing forefront of change in pupil personnel services. There 
must be an agency outside of the established institutions that can be 
ever-viable force for change to keep pupil personnel services con- 
stantly attuned to societal changes. Two, develop the means for re- 
directing and revitalizing counselor education. This is a problem 
area behind change. The means of accomplishing this cannot be re- 
lated here, but it could be done. 

PACE has proved that pupil personnel services change. The 
purpose of^this paper has been to encourage PACE to move in newer 
ways, perhaps bolder and riskier, to effect change on a large sca.^e. 
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SCIENCE 



I ^ / 

: by Glenn O. Blough— 

The increased emphasis on science education during the past 
ten years has: (1) stimulated much activity in the schools at all 

levels; and (2) raised problems, especially in the area of curricu- 
lum development, methods of instruction, education of teachers and 
use of instructional materials of all kinds. We consider here special 
emphasis on these problems as they exist on the elementary- school 

I level, since the funded projects examined deal chiefly with this in- 

I structional level. 

I Developments and Problems 

I What science content should be taught at the elementary level 

[ and how it should be organized have been, and still remains in many 

I situations, an ur jclved problem. Graded series of textbooks have 

I 

I been, and continue to be, in many schools a major factor in deter- 

mining such content. In recent years, some inroads have been made 

I 

— ^Professor of Education, University of Maryland. 
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toward the establishing of specific behaviorial objectives as a basis 
for content selection. Another guide to content selection has been the 
establishment of conceptual schemes — a modern version of generaliza- 
tions and science principles. Still another trend in curriculum de- 
velopment is a process-centered approach built around certain pro- 
cesses considered essential in learning how to learn. In this case, 
processes are considered of prime importance; learning subject 
matter is secondary. 

In addition to these rather specific developments in curricu- 
lum constructions, there are uncounted numbers of state, city, county 
and local groups engaging in curriculum-making, which use one or 
more, or some of all the developments just described. Administra- 
tors, supervisors, classroom teachers and special science teachers 
■ are involved in these projects. Many of these groups operate without 

competent leadership (i.e. , not trained in curriculum- making, not 
knowledgeable in how children learn, or without background in science 
subject matter). Consequently, the resulting publications for use of 
the teachers are not always as helpful as they might be. Much work 
remains to be done in the area of curriculum development in science 
in the elementary schools. 

While there has been, since the inception of science in the 
elementary school, considerable emphasis on problem-solving as a 
method of learning, headway has been difficult chiefly because of the 
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lack of training of elementary school teachers in problem-solving 
techniques. More recently, new language is being used to describe 
problem-solving. It is conceived of as processes of discovery, of in- 
quiry and of the use of scientific methods. This emphasis is probably 
one of the most important developments in the area of science in the 
elementary school. Helping children to learn how to learn is em- 
phasized at all levels of instruction. This development has necessi- 
tated the use of different methods of instruction, more and better 
laboratory experiences, nnuch individualized instruction, greater use 
of materials, less telling and showing by the teacher, and more re- 
sponsibility for planning and doing on the part of the learner. 

Teacher education in the area of science in the elementary 
school has always been a problem, but with the new emphases, the 
problem is even more acute. Obviously, the objectives of the pro- 
gram cannot be reached until real progress is made in this area. Some 
of the newer projects have "built-in" teacher education features. 
Teachers are not expected to carry out the plans of the projects with- 
out thorough understanding of the objectives and methods of instruc- 
tion. This is in contrast to the usual process of handing a new science 
course of study or bulletin to teachers, expecting them to translate it 
into classroom procedures without assistance. Teachers at the ele- 
mentary level are lacking in science subject matter background, as 
well as in an understanding of the nature of science itself and how to 
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teach it. Colleges and universities, and in-service education, are at- 
tempting to remedy this situation. Younger teachers appear to be 
more nearly adequate to meet this situation than those who have been 
teaching for many years. 

Much new material has been developed commercially for use in 
teaching science in the elementary school. Title III monies have been 
largely responsible for stocking elementary schools with all manner of 
equipment and apparatus. Alas, much of it remains unused, either be- 
cause it is not needed to carry out the curriculum as it presently 
exists, or because teachers are not educated in its use. While there 
is much instructional material available, due to increased emphasis on 
the area, there is likewise greater need for instruction in when, why, 
and how to use this material. 

A summary of the more recent developments in the field of 
science in the elementary school would include these features: 

• Increased emphasis on the teaching of science. 

• A trend toward a developmental program from kinder- 
garten through high school. 

• Scientists, psychologists, educators and classroom 
teachers are involved in attempting to solve the prob- 
lem. 

• Attempts are being made to combine mathematics and 
science experiences. 

• Greater emphasis is placed on the method of discovery, 
in some instances, to the exclusion of meaningful sub- 
ject matter. 
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Through large grants from the Science Foundation to such 
agencies as the American Association for the Advancement of Science, 
as well to individuals, much experimentation has gone on. Some of 
the findings are important. In many instances, the so-called discover 

ies are not new. Rather, new personnel have discovered them. They 

are, however, having considerable impact in many school systems. 



ESEA Title III and Recent Developments 
An examination of the funded projects of ESEA Title UI indicates 
in many cases, little, if any, relationship to the problems mentioned 
in the opening paragraphs of this paper. Of the 20 projects examined, 
nine are concerned chiefly with the establishment and use of planetaria 



or 



observatories, or closely related areas such as space study. Two 
deal with the study of marine life; five with planning museums in one 
form or another, including outdoor education and wildlife study, one 
with a radiation center; one with a center for dealing with science and 
industry and only two with implementation of the present scienc e £ro- 
»r»ms designed to lift the level ^ instruction, bring them up-to-date. 



and deal with other pressing problems of science teach ina. 

While It is possible to strike out toward solving some of the 
most pressing problems of science education through emphasis on 
planetaria, museums, etc. , there is little evidence that this is the 
case in most of the projects. As described, the projects have not 
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been selected because they will help to solve problems related to im- 
proved curriculum development in grades K through 12. They do not 
seem .o take into account the other problems listed earlier. Only two 
of the projects seem to consider specifically the pressing problem of 
teacher education. Many of the projects appear to deal with "fringe" 
problems, rather than the basic problems national studies have identi- 
fied as those most needing attention. The entire plan as described 
may be carried out and still leave the most pressing problems unsolved. 

Obstacles in the Way of Achievement 

There appears to be some difficulty in writing guidelines and 
project descriptions which will attract applicants and stimulate re- 
quests that strike directly at some of the most urgent problems of 
science teaching in the elementary, and to some extent, the secondary 
schools . 

More attention needs to be given to designing some evaluating 
procedures which can be used during the entire life of the project. 

Such procedures would certainly enhance the effectiveness of the pro- 
jects, since the results of the evaluations could be used to redirect the 
efforts when desirable. 

It is quite possible that readers who are responsible for recom- 
mending the grants may, in some cases, be unfamiliar with the 
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developments and problems in the area under consideration. Conse- 
qu6ntly> funds arc allocated to fringe projects. 

The uncertainty concerning the continuation of funds has, in 
many cases, resulted in indecision on the part of those responsible 
for the project and has seriously impaired progress. 

The lack of attention to evaluation has, in many cases, been 
responsible for a full realization of the potential in the various pro- 
jects. 



Future Directions 

It is suggested that the words "exemplary" and "innovative" be 
explained in the directions given to prospective project applicants. It 
is quite possible that these words may have been, in a measure, re- 
sponsible for the apparent belief that in order for a project to be 
funded, it must deal with some "fringe" project unusual in nature, 
rather than one that is on-target with respect to the most pressing 
local problems. It is quite probable that it is not the general under- 
standing that a project can be exemplary and innovative, and still deal 
with problems that are the most pressing, and in line with the more 

recent developments on a national level. 

It is suggested that Title III applicants be urged to conduct a 
survey (it may be informal, if applicants consider themselves already 
well informed about their needs) to isolate some of the most urgent 
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local problems which must be solved in order to improve the curricu- 
lum and teaching. It is further suggested that classroom teachers be 
actively involved in such needs surveys. The projects would then be 
planned to fit the problems indicated by the survey. 

A built-in teacher-training program should be a part of any 
funded project since relatively few elementary school teachers are 

adequately prepared to teach science. 

A greater emphasis on evaluation should be a part of all pro- 
jects. A plan for evaluation should be both comprehensive and speci- 
fic. It should attempt to measure the attainment of the objectives, in- 
sofar as this is possible. 

Attempts should be made within the states to determine whether 
or not the problem being considered has been given attention elsewhere, 
so that the findings could be made available and the proposed projects 
could then build on these findings. In this connection, attempts should 
be made to coordinate findings and disseminate them, thereby increas- 
ing their usefulness. 

Project readers who assist in deciding whether or not the pro- 
jects are to be funded should be chosen from persons well grounded in 
the specific area under consideration. They should also be as familiar 
as possible with the area requesting funds. Needs in one area vary 
greatly from those in others. 

Projects requesting funds should show familiarity with the work 
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of the national projects and other significant activities, and should in- 
corporate such information to show how their requests plan to delve 
deeper into the problem. The same holds for significant research in 



the field. 





SOCIAL STUDIES 



*/ 

by Dorothy M. Fraser" 

During the 1960's, the social studies program of the schools 
has been the subject of considerable study and experimentation. More 
than 60 special curriculum projects have been funded by public or pri- 
vate sources to develop plans and materials for improving the social 
studies program. In many school systems, revision efforts are under 
way. These activities have come about because of dissatisfaction with 
the results achieved through traditional social studies programs. Al- 
though there is great diversity in the nature and scope of the various 

1 / 

social studies projects, significant emphases are emerging. 



—^Coordinator of Social Science, College of Education, Hunter 
College of the City University of New York. 

-^Fuller descriptions and analyses of current developments in 
social studies will be found in: Fenton, Edwin, New Social Studi . ^ , 

Holt, Rinehart, Winston, 1967; Fraser, Dorothy M. , "Social Studies 
Education," in Curriculum Handbook for School Adm inistrators, 
American Association of School Administrators, 1967; Gibson, John 
S. , New Frontiers in the Social Studies , Citation Press, 1967, 2 vol. 
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Major ^evel^ 2 Sl£IilS. 

The need to plan within the context of the total elementary and 
secondary school social studies program in order to facilitate the 
pupil's sequential development of concepts, values and skills, is 
recognized. The use of basic concepts, generalizations and principles 
from the social sciences in order to construct an ideational framework 
for the total program is generally accepted. Social scientists and 
social studies educators have cooperated to identify the elements of 
such a framework.”^ 

Realizing that "knowledge" is subject to rapid and continuing 
revision, and that the as-yet undeveloped social science knowledge of 
the future cannot be taught today, there is a new emphasis on helping 
pupils to understand how the conclusions now accepted as "knowledge" 
have been arrived at, and how to gain command of the process of 
learning. Facts, which have too frequently been taught as isolated bits 
of information to be memorized, are to be used by pupils to develop 
concepts and generalizations which they can employ as tools for or- 
ganizing the masses of specific data they will encounter. Skills are to 
be taught functionally by having pupils apply them in social studies 

— ^See: Morrisett, Irving, ed. , Concepts and Structure in the 
New Social Science Curricula . Social Science Education Consortium, 
1966; Price, Roy A. , et al. , Major Concepts for Social Studies , 

Social Studies Curriculum Center at Syracuse University, 1965; A 
Conceptual Framework for Social Studies in Wisconsin Schools ; Wis- 
consin State Department of Public Instruction, 1967, revised edition. 
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investigations and, cumulatively, by moving from simpler to more 

complex aspects. Analyses of skill areas, identifying sub~ skills of 

• 1 / 

progressive levels of difficulty, are available. 

A second group of developments relates to the selection, or- 
ganization and grade placement of content for the social studies cur- 
riculum. Traditional programs have been criticized for using content 
that is outdated, in some cases because it does not reflect current 
scholarship, but more frequently because much of it has been irrele- 
vant to conditions and problems of today's world. To rectify this situa- 
tion, content from the social science disciplines which can contribute 
directly to an understanding of contemporary society is being drawn 
into the school program. Included are materials from economics, 
sociology and anthropology, as well as hitherto neglected aspects of 
political science and geography. Focus on critical societal problems 
is receiving new emphasis. Also, the study of cultures and nations 
other than those of the western world has been introduced rapidly. 

Traditional grade placements of content have been affected by 
the introduction of these new materials, as well as by revised views 
about children's learning. Repetitive surveys of traditional subjects 
are being replaced by "depth studies" of topics judged to be significant 
for understanding modern society. Fuller study of fewer topics 

-^Carpenter, Helen McCracken, ed. . Skijl Development in 
Social Studies . Thirty- Third Yearbook, National Council for the 
Social Studies, -1963. 
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provides the time required for inductive procedures and practice in in- 
quiry skills. As a result of experiments which have indicated that 
children possess hitherto untapped learning potential, many topics are 
being treated earlier in the K through 12 program than formerly. 

Finally, new emphases are developing with regard to instruc- 
tional strategies. Procedures that stress inquiry by the student, that 
encourage his curiosity, and that lead him to "create his own knowl- 
edge" are highlighted. The use of a range of multi- media materials, 
including the new educational technology, is replacing traditional de- 
pendence on the textbook as a single source of information. Individuali 
zation of instruction, with continuing diagnosis of the pupil's progress 
and immediate feedback into instructional planning, is recommended. 

PACE - Sponsored Social Studies Projects 
Do ESEA Title III projects dealing with social studies reflect 
these current developments in the field? A review of descriptions of 
the 66 projects which had been funded prior to September, 1967, in- 
dicates that in varying degrees they do reflect these currtent develop- 
ments. More specific information, obtained through the analysis of 16 
proposals, visits to three projects and interviews with personnel con- 
nected with other projects, confirms this conclusion. Some aspects 
of current developments in social studies are receiving much more at- 
tention than others, however. 
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Many, perhaps most of the projects, include some emphasis on 
the use of multi- media materials, and on instructional procedures in- 
tended to replace "assign-recite-test" routines with more varied activi- 
ties for pupils. One project is focused directly on the use of visual aids 
in teaching geographic concepts. Films, filmstrips, tapes. and transpar- 
encies, as well as non- textual reading materials (pamphlets, paper- 
backs, biographies, etc.), are stressed as learning sources in many 

of the programs. ^ 

A substantial number of the projects emphasizes student inves- 
tigations in the community, using local resources such as industrial 
plants, historical sites, museums, art galleries, and outdoor educa- 
tional centers, or making surveys on appropriate topics. Classroom 
visits from persons in the community with special experience or com- 
petence regarding aspects of the social studies program represent 
another way in which projects are attempting to broaden the sources of 
information available to students. In one project, dealing with the area 
of U. S. history, high school students participate with archaeologists 
and historians from nearby universities in excavating and restoring a 
pre-Revolutionary War site. 

A more flexible use of pupil and teacher time is sought in sever- 
al of the pirojects, including provision for modular scheduling and team 
teaching. Large group lectures, small discussion groups and indepen- 
dent study are provided for the students involved in these programs. 
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The development of new units and courses for various grade 
levels is underway in many of the projects, while some are engaged in 
reconstructing the social studies program for a school unit (for ex- 
ample, intermediate grades, or senior high school), and still others 
are replanning the total elementary and secondary sequence. Materials 
from several of the projects primarily focused on curriculum revision 
indicate a concern for developing a conceptual approach, in the new 
program. 

Special content emphases, such as Asian, African or Latin 
American studies, the study of world affairs, or the study of resource- 
conservation, are a central factor in about 15 percent of the projects. 
Enriching social studies courses by introducing aspects of the fine 
arts and literature is the purpose of some projects, and museum cen- 
ters and services are utilized in a number of cases. A few projects 
are devoted to developing interdisciplinary humanities courses for 
secondary school students. 

Many of the project descriptions indicate that some attention is 
being given to in-service programs for teachers, in order to acquaint 
them with new content and new instructional procedures. A very few 
project descriptions focused directly on in-service education. A favo- 
rite device is the "educational resource center" which, in most cases, 
is set up to serve a number of cooperating school districts. Demon- 
stration facilities, as well as collections of learning materials and 
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workrooms in which teachers may learn to create various types of 
audio- visuals, are included in many of the centers. 

This brief survey of emphases in PACE projects concerned 
with social studies indicates that Title m funds have indeed stimulated 
efforts to improve programs in this field. To what extent are the pro- 
jects operating on the "cutting edge" of developments in the field? The 
answer varies enormously from project to project, ranging from very 
much so" to "only slightly. " 

The projects which fall at the upper end of the scale have a num 
ber of characteristics in common. They are addressed to needs that 



have been clearly identified as having more than routine importance. 
These may be needs which exist in the specific school- community to be 
served, or more general societal needs. There is evidence that the 
planners were well-grounded in the broad professional literature of 
social studies education, and that they had searched out specific re- 
search and innovative efforts pertinent to their project. In many 
cases, several school systems have joined in a cooperative effort, 
often utilizing the device of an "educational center. " Usually, the re- 
sources of both the school systems and nearby institutions of higher 
learning are tapped, with social scientists as well as social studies 
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specialists becoming involved. 

The projects at the upper end of the scale, whether limited in 
focus or broad in scope, are being developed with a total social 
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studies program in mind so that attention can be given to the sequential 
growth of concepts and skills. Attention is given to improving both 
process and substantive content in the program. There are plans for 
interim evaluation, with feedback, as well as for final evaluation. 

Plans for dissemination of the innovation go beyond lecture-type in- 
service sessions or single visits by teachers to a demonstration cen- 
ter. Active involvement of the target population is sought at an early 
stage, and provision for continuing and broadened participation is 
built-in. If short range steps are taken to involve considerable num- 
bers of teachers immediately, such as workshops in the use of multi- 
media materials, they are planned to feed into, and not work at cross 
purposes with, the long-range purposes of the project. Lines of com- 
fxivinication are developed with parents and others in the community, 
as well as among school personnel. 

Projects falling at the lower end of the scale lack several, and 
in some cases most, of these characteristics. A common difficulty 
may be described as a piecemeal approach, in which the specific pro- 
ject has been planned as an isolated entity, rather than in relation to 
the total social studies program. This approach is likely to be as- 
sociated with the apparent lack of a broad acquaintance with current 
developments in social studies education, an inadequate diagnosis of 
the needs the project is intended to meet, poorly defined objectives, 
and the failure to search out appropriate consultant help. 
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Future Directions 



What can be leanred from the experience of PACE to date with 



social studies projects that can serve as a basis for planning for the 



future? What emphases should be encouraged, as the limited "risk 
capital" represented by Title III is invested in starting new projects., 
or continuing those already in existence? 

A number of rather obvious criteria for selecting programs to 
be funded are implied by the preceding discussion. 

1. Projects, whether broad in scope or sharply focused on 




a limited aspect of the social studies curriculum, should be designed 
within the context of a conceptual structure for the total elementary 
and secondary social studies program. 

2. The project plan should be firmly based on available find- 
ings from research and experimental programs in social studies. 
School personnel who are to take the lead in formulating the plan and 
developing the project, should utilize appropriate consultants from the 
beginning, so that maximum benefit can be obtained from their exper- 
tise. Depending on the nature of the project, these consultants might 
include social scientists, social studies specialists and other profes- 
sional experts in such fields as evaluation, multi-media materials or 
learning theory. 

3. Proj ect goals should be defined clearly, and in relation 
to the need the project is intended to meet. Measures for evaluation 
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should be an integral part of the plan. They should include provision 



for specific and frequent evaluations of procedures and materials in 



relation to project goals, with feedback to provide a basis for decision- 



making about the next steps, as well as for product evaluation at the 



conclusion of the project. 

4. Provisions for involving the target population of teachers. 



through direct participation in development of the project, and/or 



through in-service programs, should also be built into the plan. In 



formulating these provisions, current experimentation with new ap- 

4/ 



proaches to in-service education should be consulted. 

5. Cooperative projects in which school systems join together 



to attack common problems should be encouraged. Such broadly based 



projects would have the advantage of pooling resources, both fiscal and 



personnel, so that projects of greater magnitude could be undertaken 



than those possible if each district were working alone. Also, a wider 



field for the dissemination of project results would be automatically 



established. 



In addition to these general criteria for future Title III projects 



in the field of social studies, four priorities are suggested. 



1 . In funding PACE projects , priority might well be given 



to programs in which materials from experimental curriculum studies 



4/ 



— See case studies reported in PACEreport , April, 1968. 
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would be selected , adapted , and tested with various populations in the 
school setting . 

Curriculum plans and materials are now available in 
flood proportions from the major research and development projects 
which began their work some five years ago, under USOE's Project 
Social Studies program, as well as from other specially funded curric- 
ulum studies. These materials are diverse in nature and scope, rang- 
ing from single courses for a particular grade level to "wall-to-wall" 
curriculum plans. Although most of the materials have been tried out 
in school situations, during the development process, further field- 
testing is needed before their potential can be judged. Even more im- 
portant, different assumptions about learning, different bases for con- 
tent selection as well as different organizational patterns are reflected 
in the various materials. To utilize them effectively, a school system 
must select from them and adapt the materials chosen to construct a 
balanced, coherent program. Projects which could accomplish this 
task would aid not only in the dissemination of innovative materials, 
but would also help other school systems to see just how this important 
step can be taken. 

2. Pilot projects to develop and test new social studies cur - 
ricula for ghetto schools would seem to deserve high priority . 

In spite of the current concern about improving educa- 
tional opportunities for disadvantaged urban youth and the expenditure 
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of considerable Title I funds for this purpose, relatively little has been 
done to explore new types of social studies programs for this group. 
Bold efforts to adapt experimental materials to the needs of ghetto 
pupils, or to create and test drastically different approaches and ma- 
terials, are critically needed. 

3. Priority should be given to those social studies projects 
which explore ways of helping to prepare young people to cope with 
the basic conflicts existing in American society . 

The tragic riots and assassinations of the past year are 
both symptoms and results of critical problems which, if unresolved, 
threaten the future of democracy in America. The problems, so deeply 
rooted in our society, cannot be solved by the schools, but the schools 
can contribute to their alleviation. The social studies program has a 
particular responsibility in this connection. Some experimental ap- 
proaches and materials for dealing in the classroom with such topics 
as public controversy, race relations, poverty, and the role of law 
in American life have be*3n developed by special curriculum projects. 
They are too little known by social studies teachers, too little imple- 
mented in social studies classrooms. There is an urgent need to test, 
adapt and disseminate these materials. Experimentation with other 
potential approaches for helping young people to learn about basic 
conflicts in their society, to think rationally about them, and to clarify 
their own values and attitudes toward them, . is also needed. 
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4 , Proiects with major focus on new patterns ^ ^-service 

education for social studies teachers should ^ priority. 

It is generally accepted that the classroom teacher has a 
crucial role in implementing curriculum innovations. Because the 
social studies curriculum is changing so rapidly, with new emphases 
in both content and process, the problem of in-service education for 
teachers who work in the field at either the elementary or secondary 
level, represents another important frontier. The ineffectiveness of 
traditional in-service programs — after- school sessions, institute 
days, .'^nd so on — is widely recognized. As noted above, some experi- 
mentation with new arrangements for in-service education is already 
under way. Pilot projects to adapt promising patterns to the particu- 
lar problems of in-service growth in social studies teaching and to 
continue the search for other effective approaches, could bring im- 
portant advances in the improvement of social studies experiences of 

children and youth. 
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SCHOOL REORGANIZATIONAL ACTIVITIES 

9{C j 

by Maurie Hillsorr” 

Now is the time either for a fundamental reformulation of the 
directions of public education or for a revolutionary intervention 
which can redirect the process of education. There are important de- 
partures and new developments, both in elementary school reorganiza- ^ 

tion and in the educational processes. Such themes as continuous pro- 
gress education, teacher collaboration, open plan teaching, diagnostic 
and prescriptive approaches to learning, the development of sequential 
curricula based on skills, concepts, or inquiry, and modular scheduling, 
not to mention the concomitant activities of independent Study, repre- | 

sent a shortbut nonexhaustive list of the concerns and actions currently 
taking place. Many of these concerns are derived from very plausible | 

points of view concerning the way a learner learns, i. e. , the process 
of education. Even though there presently are no individual models 
which include all these areas in a completely developed way, at least 



-'^Professor of Education, Graduate School of Education, Rut- 
gers - The State University of New Jersey. 
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we can see that the research of the past years concerning child growth 
and development, together with the whole scientific revolution concern- 
ing learning, have indicated that elementary education must certainly 
reject many of the items now constraining the education of the child. 
Both research and practice point to needed activities for elaborating 
the abilities or insights of children to a point where they actually par- 
ticipate more fully in the educative act. The themes noted above have 
been put into operation in varying degrees in several places throughout 
the United States. PACE has supported many proposals in which these 
themes have been planned as germane aspects of the educational inno- 
vation. 

However, verbal attestation does not always represent fruitful 
and meaningful implementation, and a tremendous amount of lip-ser- 
vice has been given to these themes. There are, embarrassingly, 
many places which refer to their programs as nongraded schools. On 
closer investigation, they are frequently found to be merely recastings 
of the graded school in a different, but equally constrained, format. 

The major concerns of a diagnosis and prescription, the development 
of learning sequences, and the application of teaching strategies, using 
the specified styles and needs of the youngster's individual learning 
timetable as the yardstick, are absent from these programs. Indeed, 
they do not make any basic contribution at all to the growth of the chil- 
dren involved. 
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ESEA Title III Projects reflect many innovative themes, and 
the proposals themselves offer insights into what should be considered. 
The proposals which concern themselves with continuous progress, 
nongraded, collaboratively taught, open plan diagnostic and prescrip- 
tive approaches are many. Attempts to break with the customary 
graded school organization, moving with the new organizational ve- 
hicles. seem to be rather intensive. Many of these projects have 
been funded and are operable. The unfortunate situation is, however, 
that so few of them actually move from verbal proposal stage into 
implementation activity so necessary and vital in establishing the 
value of an educational innovation. 

Obstacles 

There are many obstacles which prevent the establishment and 
implementation of programs. Chief among these is the lack, of oppor- 
tunity for educational training programs as well as the "lead-in" time 
necessary to alter the perceptions and techniques of those involved in 
the implementation of innovative approaches. Rather than lamenting 
the problems which constrain programs of implementation, criteria 
for action and forceful programs of implementation should be estab- 
lished. If the future of PACE is to see a move from verbal testament 
to programs of actual implementation, then some set of items ensuring 
that implementation must indeed be found. 
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There are certain necessary elements of implementation in- 
herent in achieving a process leading to the adoption of various themes 
of organizational innovation. These must be addressed if the establish- 
ment of new forms to allow for a different kind of education is to be 
defined and created. 

If the themes of nongrading, collaboration, individualization, 
and other innovations are to be realized, then, first, there must be 
created a school organization for flexibility. An attack must be made 
on the fundamental constraints of the present organizational patterns 
of the elementary school. Mechanical schedules must differ. An at- 
tempt must be made to create more effective teacher utilization by de- 
termining teacher strengths and by allowing for, if not indeed insisting 
upon, creative teacher -pupil deployments for the betterment of educa- 
tional programming. There must be a set of activities to lead teachers 
into a program for becoming diagnostically and prescriptively able. 

The whole concept of diagnostic and prescriptive education must, of 
necessity, be embraced, since these are underpinnings of the whole 
activity of educational improvement. Teachers must be trained in 
the techniques of establishing learning sequences beginning with the 
very earliest pre-K years, and continuing through all the school years. 
They must create diagnostic tools appropriately related to these se- 
quences. They must create inventories. There must be a program of 
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diagnostic feedback in order to establish a program of individualiza- 
tion leading to achieved competency. 

Other items which must be addressed, if there is to be any fu- 
ture at all for the whole school reorganizational program, quotes are 
those activities which lead to a greater sense of collaboration among 
teachers, and which, at the same time, draw from that collaboration 
a greater commitment to the individualization of pupil instruction. The 
suggestions to be found in many of the proposals (as well as in the 
operable programs for collaboration) are never truly established. 

There is a seeming inability to create the mechanical schedules within 
the school for developing the techniques, approaches and strategies for 
teacher collaboration. Teacher collaboration, especially in the area 
of planning, is a fundamental necessity if there is to be a well thought- 
out, innovative reorganizational program in a school. It is through the 
collaborative aspects of planning that pupil redeployment and teacher 
assignments can be made in a more fruitful way. Collaboration brings 
more specifically worked-oqt assignments which address real prob- 
lems and real concerns. It is through their collaborative endeavors 
that teachers can select appropriate and pertinent material, and es- 
tablish the criteria for educational performance. It is through these 
collaborative arrangements that teachers can revise curricula and 
create opportunities for various sizes of group instructional activities 
of value in addressing the learner. It is through the collaborative 
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endeavor that teachers can establish programs individual in nature, 
which can lead to a greater commitment on the part of the learner. | 

I: 

These same programs lead to individual contract learning where ! 

teacher and pupil, in a one-to-one contractual relationship, pursue 
an active program of learning. These are the areas in which pupil- 
team learning programs and programmed approaches are created, 

Ij 

wherein teachers serve as "guides** for the learning activities. They, | 

along with their pupil- teams, can serve one another in the oral-prac- | 

tice programs and individual learning pursuits which lead to compe- || 

tence in learning. 

The basic needs, needs of major dimensions, are activities, 
attendant supports and the various behaviors that lead to a continuous 
implementation of a program. 



Future Directions 

The directions for PACE should be those aimed at creating a 

readiness on the part of the teacher to become diagnostically and pre- 

* * 

scriptively oriented. PACE should aim at endeavors which carefully 
assess the dimensions of teacher readiness, and then establish pro- 
grams to deal with particular weaknesses, so that when teachers move 
into a collaborative situation and diagnostic and prescriptive teaching, 
they can do so with the kind of ipsight needed to cope with the problems 
facing their learners. 
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If any real commitment toward change is to be gained, PACE 



should embark on a program aimed at ensuring the mounting of an in- 



tensive set of activities designed to alter mechanical organizational 



vehicles within the schools. Teachers must be able to plan together 



within the school day. They must be able to offer their efforts in a 



continuing way, and to make decisions allowing them to expand their 



services of instruction to their young charges. The alteration of the 



self-contained classroom; the creation of vehicles to ensure the col- 



laboration of teachers, and the actual implementation of these activi- 



ties are necessary requisites for the creation of a new era of educa- 



tional intervention leading to higher levels of intellection, as well as 



human forbearance. 



The major obstacles to change are seen in the fundamental in- 



abilities of people. People unwilling to move, because of these in- 



abilities or insecurities, need to be worked with. The support of pro- 



grams should be contingent upon education of people so that they can 



have a major involvement in real educational innovation. 



Taxonomies of readiness for implementation need to be estab- 



lished at two levels of operation, first, at the classroom level and then 



schoolwide. Programs designed to advance creativity in education in 



the elementary schools of the country, where innovative themes are con- 



cerned, are, at this reading, still tentative in nature. Implementation 



of some of the exciting concepts are nonexistent. This is lamentable 
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but correctable. The correction requires a more forceful set of 
criteria, criteria which are implementation-oriented. Supportive ac- 
tivities to assure implementation need to be built into all the proposals 
and evaluations of programs. Energies should be aimed at action 
orientation, rather than over- emphasizing the verbal enunciation of 
ideas. Minimizing the length of time for studying a program type and 
maximizing those activities concerned with strategies for operation and 
action would seem appropriate directions to take. Commitment to ac- 
tion is essential. PACE could then have a double meaning in its 
acronym. It could stand for Programs of Action in Creative Education. 
Action is certainly essential, and long overdue. Any attempts to 
hasten, support, and sustain action can only reap a more meaningful 
harvest of educational growth and success. 
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THE GIFTED AND THE DISADVANTAGED 



*/ 

by A. Harry Pas sow” 



As one looks at the current scene in American cities and im- 
poverished rural areas and at the plight of the disadvantaged and the 
gifted in such settings, any progress is difficult to discern. The re- 
cent report of the National Advisory Commission on Civil Rights ob- 
serves: 

None of us can escape the consequences of the 
continuing economic and social decay of the central city 
and the closely related problem of rural poverty. The 
convergence of these conditions in the racial ghetto and 
the resulting discontent and disruption threaten demo- 
cratic values fundamental to our progress as a free 
society.—/ 

Speaking specifically about education, the Commission con- 
cludes that "for many minorities, and particularly for the children of 
the racial ghetto, the schools have failed to provide the educational 



—/professor of Education, Teachers College, Columbia Uni- 
versity. 

—^National Advisory Commission on Civil Disorders, Report _qf 
the National Advisory Commission on Civil Disorders, New York: 
Bantam Books, Inc. , 1968, p. 410. 
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experience which could help overcome the effects of discrimination 
and deprivation. Further, it believes that the "bleak record of 

public education for ghetto children is growing worse. " The factors 
which converge to produce this deteriorating situation are cited as: 
growing racial and economic segregationj inadequately prepared staff 
with high turnover rates; overcrowded and inadequately supplied 
schools; obsplete and poorly equipped facilities; inappropriate cur- 
riculum and materials; insufficient funds; poor community- school 

n 

relations and overall debilitating effects of ghetto environments. 

A similar analysis, containing many of these same factors, could be 
made of education in the depressed rural areas. 

To reverse present trends and to move toward the provision 
of full equality of educational opportunity for the disadvantaged, the 
Commission recommends the pursuit of four "basic strategies 



, 11 . 



(a) inrrftasing efforts to eliminate de facto segregation, (b) provid- 
ing quality education for ghetto schools, (c) improving community- 
school relations, and (d) expanding opportunities for higher and voca- 
tional education. The specific programs under each of these strategies, 

suggested by the Commission, have short titles not unlike those found 

4/ 

in a listing of Title HI PACE projects dealing with the disadvantaged. 
-^ Ibid . . pp. 424-425. 

~^Ibid. , pp. 426-438. 

-^Ibid. , pp. 440-454. 
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For example, among the illustrative programs suggested by the Com 
mission, are: 



Exemplary city or metropolitan schools offering special 
courses and programs to attract students of varying 
social and economic backgrounds. 

Educational parks. 

Improving the quality of teaching in ghetto schools. 

Year-round education for disadvantaged students. 

Improving educational practices in elementary and 
secondary schools. 

An intensive national program to increase the verbal 
skills of ghetto residents. 

Expanded experimentation, research and evaluation 
of effective methods for teaching disadvantaged 
pupils. 

Elimination of obstacles to community participation 
in the educational process. 

Developing school facilities to serve the educational 
and other needs of the total community. 

Use of local residents as teacher aides and tutors. 

Expansion of Upward Bound , and the establishment 
of special one-year postgraduate college-pre- 
paratory schools. 

Linking vocational training programs to job oppor- 
tunities. 

Expanding part-time cooperative education and work- 
study programs. 

A compilation of Title I projects would overlap such a listing 
as well, although such programs tend to fall into a limited number of 
more conventional action projects dealing with remediation (especially 
in reading and the other basic skills), early childhood programs and 
enrichment opportunities. 

The stated purpose of Title I is ”to expand and improve. . . edu- 
cational programs. . .which contribute particularly to meeting the 
general educational needs of the educationally deprived. " PACE 
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projects, on the other hand, are presumably designed to supplement 
existing programs and facilities, and to create innovative solutions 
to persistent problems. If the Commission’s recommendations are 
considered as "the cutting edge" for developments in the field of the 
disadvantaged, many Title III projects do involve the programs and 
services advocated. And yet, such an observation must be couched 
with reservations. 

The chief reservation might be stated in terms of form and 
substance. It is possible to operate an early childhood program which 
has all of the form but none of the substance which could make a real 
difference in the lives and futures of ghetto children. It is possible to 
install a variety of new media and instructional materials which have 
no more relevance to minority children than did the old resources. 

It is possible to flood a school with new, and supposedly more ap- 
propriate, curricular materials, without providing for the education 
of the staff in terms of knowledge, insights, attitudes and values es- 
sential for the meaningful use of such materials. 

In short, unless adequate attention is given to the critical ele- 
ments of programs which are substantive, little real difference is 
likely to occur in the development of the disadvantaged pupil popula- 
tion. Since PACE programs are intended to be innovative as well as 
exemplary in nature, the requirement of a clear rationale for the 
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inclusion of particular elements or components in a proposal would 
not seem unreasonable. 

In making its recommendations, the National Advisory Com- 
mission on Civil Disorders suggests no priorities for the basic stra- 
tegies it urges city school systems to pursue. It could be that educa- 
tion in the ghetto, and to a lesser extent in depressed area rural 
schools, is in such a state that programs must be developed; in all 
four areas simultaneously, or that well-designed and carefully planned 
programs in any area could have a significant impact. However, the 
cafeteria- eclectic approach has evidenced severe limitations in the 
past, and there is little to suggest any greater promise for the future. 

As for the education of the gifted, the past few years have 
witnessed a sharp drop in activity and program planning. To some ex- 
tent, there has been a consolidation of what has been learned from the 
research and planning of the dozen years or so since the beginning of 
the "third wave of interest" in the 1950’ s. However, some of the prac- 
tices and programs developed for the gifted and the talented seemed 
incompatible with mounting pressures to provide for the disadvantaged 
and the racially isolated. Both concerns — education of the gifted and of 
the disadvantaged — can be seen as aimed at identifying and unlocking 
individual potential. However, school systems seem to find many 
provisions in conflict with one another, with the result that programs 



" '""o " 

ERIC 












1 I 



- 87 - 

for the gifted have either been eliminated or substantially curbed in 
city school systems. 

Few school systems have really tackled the problems of iden- 
tifying and providing for the gifted and the talented among disadvan- 
taged pupil populations. Since existing instruments and procedures 
tend to miss potentially gifted individuals among the disadvantaged, 
few programs have been developed for nurturing the talented. Perhaps 
the city school systems are so overwhelmed by the magnitude of the 
educational problems to be faced in their inner-city schools that the 
development of an appropriate provision for the gifted and talented 
seems to be not only peripheral, but a luxury as well. 

As long as school systems interpret educational provisions for 

the gifted as meaning programs for the white, middle-class, moti- 
vated youngster, there will be a decline in such programs and a neg- 
lect of talent development efforts for both the advantaged and the dis- 
advantaged. The consequence of this decline in provisions for the 
gifted, and the neglect in tackling the problem of talent development 
among the disadvantaged, will undoubtedly lead to an intensification of 
the problems of the ui.,an school systems — continued white and middle 
class flight from the city public schools, continued atrophy of talent 
potential, and a general dissatisfaction with the quality of education. 

Some time ago, McClelland hypothesized that "talent potential 
may be fairly widespread, a characteristic which can be transformed 












into actually talented performance by various sorts of the right kinds 
of education." The really innovative efforts, the cutting edge, may 
well be those programs designed to increase understanding of the 
meaning of "the right kinds of education" — with individualization and 
differentiation as instructional concepts rather than organizational 
forms. As pointed out in an earlier report, education of the gifted as 
well as of the disadvantaged, must deal with the basic structure of the 
educational enterprise as a whole-” including content, sequence, me~ 
thods, resources, staffing, relationships, et cetera. 

Obstacles to Progress 

The lack of progress in upgrading education in central city 
schools has highlighted the web^ of interrelationships of which educa- 
tion is a part. Political power, housing, employment, welfare, trans- 
portation and health are all parts of the inner-city condition which im- 
pinge on the educational process. An obstacle to adequate Title III 
planning may well be the lack of real involvement of pertinent sectors 
in the planning process and in dove-tailing programs. While the school 
must serve as the catalyst for comprehensive planning, other agencies 
must be brought in from the beginning. The involvement of other cul- 
tural and educational resources in all appropriate aspects of planning 
and conducting project activities of Title III programs, must include 
key social, welfare, economic, health and political groups where 
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appropriate. For many inner-city programs, liaison with the welfare 
department, the health clinic, the day care center or a potential source 
of training and employment may be far more crucial than with a library, 
a museum or a zoo. Such involvement must be real, and it must com- 
mence in the early stages of planning. In the initial stages, as school 
personnel and others gain experience in planning together, considerable 
difficulty can be anticipated. However, these difficulties must be re- 
solved, and not by abandoning joint planning activities. 

Another obstacle to planning is the failure to assess needs and 
to establish program priorities. It is often argued that the inner-city 
and isolated rural schools face such myriad problems that wherever 
they take hold could conceivably make a difference. However, suffi- 
cient research and experience are available to suggest that some areas 
have greater pay-off promise than do others — areas such as new ap- 
proaches to staff training and development; building relevant and 
meaningful curricula; developing close and continuing relationships 
between education, training and employment; and parent education 
combined with early childhood programs. Similarly, priority assess- 
ment would provide more adequate criteria in determining what should 
be included in educational service centers designed to supplement 
existing programs and facilities. Visibility and novelty alone are not 
sufficient criteria for deciding what to supplement, and how. 

The past few years have witnessed the emergence of new 
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conditions which place at stake the future of urban education and of 
urban societies. The "Riots Commission" Report merely detailed the 
seriousness of the situation in America’s cities, and confirmed the 
critical need for intelligent action now. The report, together with 
other studies (e. g. , The Coleman Report, the Civil Rights Commis- 
sion report on racial isolation, and the Study of the Washington, D. C. 
Public Schools), points to the need for attacking on many fronts. 

Quite obviously. Title III PACE project alone will not determine 
whether the deteriorating condition of America’s cities can be halted. 
However, as the most flexible and open-ended of the ESEA programs, 
PACE could have a significaut Inipact if city school systems use Title 
III resources to develop more imaginative solutions to urban prob- 
lems. Rural areas, capitalizing on the provisions for supplementary 
services and centers, could multiply and upgrade programs for their 
students. 

Of the six PACE priorities set forth last fall, five were di- 
rectly related to the education pf the disadvantaged and the gifted. 
These included projects to improve education in core cities, to pro- 
vide early childhood education, to individualize instruction, to provide 
quality education for minority g^rppps, and to improve education in 
isolated schools. These priorities can be justified and supported on 
state as well as national bases. In fact, a strong case can be made 
for the restriction of PAQE funding to proposals in these priority 
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areas, and for the establishment of detailed priorities within each of 
these areas. 

There are political implications for such a proposal for it 
would call for a policy of unequal funding, of "positive discrimination" 
in favor of the cities and the isolated rural areas, and of allocation of 
relatively restricted resources to the most critical areas. What has 
been called "the agony of the cities" is, in reality, America's crisis. 
By concentrating on the urban problem "whole" — not just in terms of 
the ghetto of the inner-city — and by focusing innovative energies and 
personnel resources on these problems, the potential impact of the 
schools will be enhanced. 

Future Directions 

■ I ' “ 

Thus, the suggestion is made that the future direction of PACE 
be focused on the urban and the impoverished rural areas. Within 
this framework, the critical problems which must be dealt with in- 
clude: 

a. Staff development . The recruitment, training, induction 
and continuing education of personnel for all types of work (profes- 
sional, paraprofessional and volunteer) in urban and isolated rural 
areas. New and different relationships are needed between school 
systems and institutions of higher education. 
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b. Curriculum development . The building of relevant pro- 
grams and instructional processes to cope with the conditions of ur- 
banization and isolation. At the macro- level, instructional systems 
are needed which will provide for diagnosis and individualized learning 
systems, which will provide for both cognitive and affective growth, 
and which will giva meaning to the expression of "quality of oppor- 
tunity. " 

c. Community development . Attacking urban problems by 
combining the efforts of various agencies and institutions from public 
and private sectors. Unless joint ventures can mobilize the resources 
needed to cope with comniunity problems, the inroads made by any 
single agency will remaio restricted. The difficulties encountered by 
OEO Community Action Programs poipt up both the need, and the 
barriers to, community planning. Historically, and especially in the 
inner- cities, schools have tended to keep themselves aloof from their 
communities. They must now learn how to function in a meaningful 
way . 

d. Comprehensive and continuing programs . Concern for 
the disadvantaged has resulted in the widespread acceptance of early 
childhood education as part of the total educational responsibility. 
Articulated programs are needed at all levels. However, urban 
schools have been particularly weak in the areas of vocational educa- 
tion, manpower training and redevelopment, continuing education for 
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youth and ^.dults, and parent and family education. For adults, gain- 
ful employment and acculturation to urban living are key needs for 
which the schools, in concert with other agencies and institutions, 
must make ave^ilable more adequate provisions and services. 

The above suggestions for future directions of PACE are not 
exhaustive, of course. They are not unlike those of the Advisory 
Commission on Civil Disorders, for that group accurately described 
the educational task of the urban school in the context of larger social 
ills, and correctly linked the problems of the inner-city with those of 
the impoverished rural regions. Yot, if the cities are to survive, 
planning must be comprehensive. The urban population in schools is 
not simply an agglomeration of impoverished, racially isolated youth. 
The urban schools must educate all pupils with all their ranges of dif- 
ferences in ability, interest and motivations. Unless provision is ri: 
made for the widest range, from the very able, highly motivated, self- 
assured pupil to the less able, apathetic or hostile, self-denigrating 
child — the result will be an intensification of the present downward 
spiral. 

The "band-aid" approach to the problems of the city has simply 
not worked. What are needed now are creative and innovative ap- 
roaches — new and different solutions to old problems. This, of 
course, is precisely the goal of PACE. If, in the new framework, in- 
dividual states and consortia bring new ways of planning Title III 
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programs, new means for sharing and exchanging insights and experi- 
ences, and new techniques for dissemination, then PACE will have an 
impact commensurate with its potential. There are many obstacles 
to the realization of these things: personnel, bureaucratic rigidity, 
the freezing of communication channels, selfish concerns, and so 
forth. However, the emergence of the few highly creative, substan- 
tive proposals and programs in the first two years of PACE is suffi- 
cient basis for optimism for the future. 
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The important developments in the area of ’’classroom per- 
spectives” are not new ones. They are ideas that teachers have 
thought about, and some have known them well. They are ideas con- 
cerned with the kind of education designed to help each person become 
self-directing, self- realized. They facilitate competency in indivi- 
duals, and aim at developing in every child a confidence in his ability 
to be self-directing as well as in his ability to deal well with others. 

The creation of this kind of learning atmosphere is a major responsi- 
bility of the teacher. Classrooms should promote the ’’becoming pro- 
cess, ” giving an adequate lif^ space to each individual. 

The education of children in the school is changing. Now, it is 
necessary for teachers to have control of the so-called process goals 
which help students to acquire, interpret, evaluate, and exchange ideas. 
Raising education's aims for the child to the level of consciousness of 

-^Teacher, Chapman Elementary School, Portland, Oregon. 
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the child himself is something teachers should do. Unless a youngster 
understands the purpose of school, and learns to accept it, it must 
often seem an interference with his privacy. 

No learning is easy, and nothing we can do will make it auto- 
matic. But if we take each child into our confidence, he will apply 
greater effort in teaching the goals he sees as important. In this 
changing world where lifelong learning is indispensible to success, 
perhaps the most important thing we can teach a child is how to go 
about his work and how to do it well. 

Important Developments 

It should be noted that LEARNING is a change in behavior, re- 
sulting from the interaction of the organism with its environment. 
Learning is dependent upon activity or special training, and in this 
sense, it differs from behavior change which is due solely to matura- 
tion. Therefore, all children: 

• Can learn and want to learn. 

• Can learn to be self-directing. 

• Learn best what is meaningful and important 

to them. 

• Can learn to use library and resource center 

facilities. 

Since TEACHING is the creation of a learning environment, it is, 
therefore, necessary that teachers: 
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• Use individualized instruction geared to the inter- 

ests and needs of the children. 

• Become directors of learning, helping children 

with their tasks at hand. 

• Develop a passion for learning, since apparently 

boys and girls will have to go on learning, 
unlearning and relearning all of their lives. 

PACE Developments 

Of the Title III programs examined, this consultant found: 

1. Some projects are using a continuous evaluation to deter- 
mine stages of learning and amounts of progress. Also evident is the 
adjustment in purposes and planning of the next steps as the learners 

proceed. 

2. Some projects are designed for learners to become in- 
volved with what is important and meaningful to them. Emphasis is 
placed on individualized instruction, where a person can become com- 
mitted to something he sees of value to him. 

3. Some projects are distinctly showing the contrast between 

two approaches to learning: learning as growth, and learning as ac- 

quisition. Most program rationales can be identified as including 
both approaches. 

4. Some projects are concerned with the skills of self- 
evaluation and self-direction, recognizing that the attitudes which 
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are acquired along with the subject matter may be even more signifi- 
cant than the subject matter itself. 

Major Obstacles 

Assuming that at least some PACE projects do not reflect the 
’’cutting edge" dimension, what are the major obstacles which prevent 

this achievement? Suggested major ones are: 

1. Programs operating with emphasis on outcome, achieve- 
ment and expected conformity. The teachers' efforts are directed to- 
ward bringing students to pre-determined goals. 

2. Learning considered as an acquisitional activity. For 
some, learning and acquisition were practically synonymous. Learn- 
ing in a conventional sense required giving evidence of memorized ma- 
terial. 

3. Many obstacles are the result of attempts to reach the in- 
dividual as a person. Operating under this precept, these schemes do 
not allow learners to make decisions or choices. In no way are inter- 
ests, needs and concerns of much importance. These programs are in- 
dividualized only ^n the sense that those involved may pursue what is 
expected of them, at their own rate, in their own way. 

4. Responsibility for learning is held by the teacher or the 
"classroom manager. " It is not transferred to the students at whom 



the instruction is aimed. For these students, the educational experi~ 
ence may seem remote, often obtrusive. 

Future Directions 

It would appear that future directions for PACE should include: 

1. The availability, upon request, of personnel from the 
U. S. Office of Education to help school districts (specifically, the 
smaller ones, without staff or funds enough) to try to solve their 
problems imaginatively. If we are to find a balanced distribution of 
state Title III funds, this availability is imperative. 

2. The release of information discussing current Title III 
projects, pointing out the specific goals and objectives, along with 
some of the evaluation techniques. Information on the actual proce- 
dures utilized would be helpful. A great deal more dissemination of 
information on project results, emphasizing their strengths and weak- 
nesses, successes and failures, should be made available to those who 
might benefit from such knowledge. 

Recommended: 

a. PACE periodicals available in larger quantities. 

b. Documentary films discussing Title III in relation 
to effecting educational change. 

3. Requiring state Title III coordinators to attend the Office 
of Education's briefings on current PACE developments. The place- 
ment of emphasis on the operating programs, giving both an analysis 
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and interpretation, along with a discussion of pertinent data from 
terminated programs. 

4. The development of evaluation guidelines to assist 
those who are operating Title III projects. Specific suggestions ex- 
plaining the use of coudnuous evaluation, showing how this feedback 
can be used as both guide and directive toward further planning. Such 
a guideline could be prepared by the USOE or by another agency. 




COMPUTER TECHNOLOGY 



*/ 

by Don Bushnelh" 

The application of computer technology to educational programs 
may be one of the least unmixed blessings of this technical age. The 
potentially enormous benefits of rapid, convenient data storage and 
retrieval must be balanced by the computer technologists' demands for 
standardization and programmed control over the learning environ- 
ment. Computer-assisted instruction could individualize the learning 
process; but as it is being applied today, it is regimenting instruction 
and pigeon-holing the learner. 

Computer Assisted Instruction (CAI), as it is overwhelmingly 
applied in the few major school districts that can afford it, is an 
automated means of dispensing information in the drill and practice 
mode of instruction. Students type answers to a programmed instruc- 
tional sequence or respond to multiple choice selections in a paced 
presentation of spelling or word lists. The computer matches these 

^ / 

— Vice President and Director, Research and Development, 
Brooks Foundation, Santa Barbara, California. 
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responses with a string of correct words stored in its memory. If 
there is a match, or a suitable synonymn is found, the computer re- 
sponds with an appropriate reply and the sequence of operation pro- 
ceeds. The pedagogical model here is the standard teacher operation 
of test and retest in a fully programmed atmosphere in which all con- 
ditions of the stimulus and response cycle are anticipated (or thought 
to have been anticipated) by the programmer or instructor. 

As David Stansfield states in a recent article in Educational 
Technology , "the chief weakness of any type of automated instruction 
is that it cannot cater to unanticipated responses from the student. If 
the student, quite reasonably, responds to the question, "What do cars 



run on?" with the word "roads" or "wheels" or even "faith" or "credit 



n 



(instead of "petroleum") or whatever it is, all the computer will be 
ablo to say is "Your answer is wrong, try again. " 

The computer could be programmed with every possible re- 
sponse the student might be anticipated to make, but this would use up 
memory in short order. The basic point here is that with the relative 
inflexibility of the systems being applied to CAI, what can CAI offer 
our schools in the present that merits support through PACE? The 
answer would have to be not much. It is my opinion that CAI, in its 
present form, should be kept on an experimental basis for the next 
few years. 



In a recent issue of the Review of Educational Research . Karl 
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Zinn describes basically three modes for using the computer in the 
instruction of public school students. The drill and practice mode has 
already been described above in which the student can only do what the 
program allows. Zinn comments that if, in this mode, economic 
criteria are important, users of computer-based drill exercises must 
also consider alternative ways of achieving their objectives for student 
learning. Skills might be acquired more efficiently through paper-and- 
pencil exercises, or more pleasantly through interaction with other 
students, or more effectively as side benefits of more complicated 
problem solving tasks aided by the computer. 

Future Developments 

Language instruction in the drill and practice mode of CAI 
does hold promise for future PACE support — or at least it should be 
kept under close surveillance as it develops. In two projects that 
come immediately to mind, the power of the digital computer is being 
put to proper use even though the rigidity of the course of instruction 
is still present. As reported by Zinn, the State University of New 
York at Stony Brook is using its computer to provide exercises in 
aural discrimination, vocal reproduction, aural comprehension, and 
writing. Typed responses are handled automatically, vocal responses 
are replayed, and a self-rating by the student is used in the decision 
algorithm. The curriculum designers did not intend the materials to 
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provide a complete coarse; the students meet for a few hours of 
classroom instruction with a language teacher. This instruction is 
then supplemented by a few hours in the CAI laboratory. All drill is 
provided by CAI which allows for classroom time to be devoted to oral 
practice. 

At the University of Michigan a student learning station was 
especially designed around a small computer for use in training 
prosodic characteristics of speech. The automated system is pro- 
grammed to assist the student with three aspects of pronunciation: in- 

tonation, emphasis and spacing of syllables. The device plays a model 
word or phrase as spoken by a native- language instructor and pre- 
viously recorded on magnetic tape. The student responds as in the 
usual language laboratory, but as he does so the characteristics of his 
pronunciation are compared with those of the record model just played 
for him. The extent and direction of discrepancy in pitch, loudness or 
tempo are displayed immediately for him on a meter device that regis- 
ters in the center of a dial when his pronunciation is perfect, and de- 
viates to the left or right if his pitch is too low or too high, his em- 
phasis too faint or too strong and his pacing of syllables too slow or 
too fast. These two developments in creative application of the com- 
puter will someday serve as good adjuncts to the standard language 
laboratory. But before such applications are available to PACE Cen- 
ters, they must be tested and retested in the university laboratory. 
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As a summing up of CAI and its application in real school 
settings, it is appropriate to quote Zinn directly: "The benefits uni- 

que to computer presentation of text and test material have not yet 
been demonstrated. . .Few of the computer-based lessons have made 
much use of capabilities beyond those which can be accomplished with 
printed formats for tests and programmed instruction. In these cases, 
the computer may have played a significant role in improving instruc- 
tion by helping the author through more careful organization, testing, 
and revision of materials. However, in the end his self-instruction 
package may be presented to students almost as effectively (and with 
considerably less time and cost) in booklets and audio-visual modules." 

Vocational education is an area of promise for PACE. Automa 
tion is creating many new national problems in employment. There is 
a fast growing need for appropriate automated vocational training 
techniques in the future. Semi- automation in vocational education and 
dynamic simulation for on-the-job training are now technically feasible. 

A number of innovative, intensive training programs have 
evolved from research with manned space-craft. Men have been placed 
in intimate operational contact with larger, computer-centered, infor- 
mation processing systems (called simulator trainers) in extremely 

complex task environments. In such environments, much has been 

learned about the human capacity for absorbing information and making 
quick decisions. Simulator trainers will increasingly be able to sense 
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human actions as well as to present alternative programmed learning 
situations. 

Among the immediate vocational training applications of such 
complex equipment are electronic maintenance, automated industrial 
operations, automated warehouse handling, and similar occupations. 
Vocational training programs in the PACE high schools should be 
designed to provide terminal skills training for specific jobs as well 
as vocational experiences that can amplify purely academic and pre- 
professional courses. 

Scheduling is another future direction. Even in a traditional 
school program, manual scheduling has become increasingly complex 
and time consuming. The educational data processing system can 
greatly facilitate master scheduling and student assignment. Two 
families of computer programs have been built for generating master 
schedules that encourage curricular innovation and accommodate the 
needs of individual students. These are the General Academic Simula- 
tion Programs (GASP), developed at Massachusetts Institute of Tech- 
nology, and the Stanford School Scheduling System (SSSS) , devised at 
Stanford University. Both of these scheduling approaches have been 
tested in over 100 school systems and are undergoing continual refine- 
ment. While specific applications of flexible scheduling programs 
have been greatly increased under PACE, future applications of tech- 
nologically aided scheduling should continue to be increased. 
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Tr=,^ilitiBs Planning holds much promise for the future. The in- 
troduction of computer technoiogy can expedite improvements in physi- 
cal facilities. Computer modeling of future school buildings can be 
based on up-to-date information about population trends, projected 
finances, curricular aims. etc. Even in existing school buildings, the 
computer can aid in studies of room utilization, traffic patterns, at- 
tendance to displays, etc. . so that existing facilities can be put to 
more effective use. A model of the Denver School System has been 
recently announced as available from the Colorado State Department 

of Education. 

In summary, it should be stated that the computer, through 
PACE support, could become one of the unique tools in the deposit y 
of instructional technology, to free the student and the educator from 
the shackles of a too rigid system. What is being proposed is a means 
to let the technology revolutionize the educational system, not rein- 

force it. 
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SUPPLEMENTARY SERVICES CENTERS 



*/ 

by Ira J. Singer" 

A new era in the life of Title III is upon us. The future of Title 
III as a generator of educational atomic energy has not been entrusted 
to State Advisory Councils and educational agencies. 

This new responsibility of the states is clearly described in 
preliminary recommendations for the design of a state plan, published 
by the Office of Education earlier this year. After defining the func- 
tions of the State Advisory Council as advisory to the state educational 
agency "in the preparation and administration of the state plan, and 
in the development of criteria for approval of applications under the 
state plan," the Office of Education document goes on to define as 
further functions of the advisory council, the "review and recommenda 
tion of all applications for grants under the state plan. . .the annual 
evaluations of all projects funded under the state plan. . .an annual re- 
port by the state advisory council of its activities, recommendations, 

■—^Assistant Superintendent, West Hartford Public Schools. 
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and evaluations under the state plan to the U. S. Commissioner of 
Education and to the National Advisory Council. " The preliminary 
guidelines also provide that the state educational agency shall appoint 
the advisory council, employ the staff "to be utilized by the State Ad- 
visory Council in executing its responsibilities," design provisions 
for review, recommendations and evaluations of all projects by the ad 
visory council, and administer and supervise all programs operated 

, i/ 

under the state plan. 

With the shifting of Title III power and funds from federal to 
state agencies, it is now the obligation of the latter to execute faith- 
fully the initial intent of Title lU, recently restated as follows by the 

Title III National Study Team: 

1. to provide venture capital which the nation's (state's) 

schools could draw upon to experiment with new ways to meet old and 
new needs, and to find better ways of performing their customary 

services; 

2. to encourage the establishment of supplementary educa- 
tion centers of particular relevance to the central cit es, 

3. to encourage the 50 states and territories and their 
22. 000 school boards to form neighboring consortiums and regional 

-^Regulations Pursuant to Title m of the Elementary and 
Secondary Act of 1965, U. S. Department of Health, Education and 
Welfare, Office of Education, Draft Copy, February, 1968, p. 8. 
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a.rra.ng6iTientS| to pool bettor their resources to s.tta.ck common prob- 
lems; 

4. to assist the schools in mounting demonstrations of 

promising new practicOi to conduct them, and when finding them good, 

2 / 

to make them broadly known.” 

Previous reports published by members of the Title III National 
Study Team analyzed the substance etnd quality of Title III projects 
funded during the period of federal administration. There is general 
agreement among the panelists that Title III is both essential and 
promising. While one can score the lack of truly inventive proposals 
during the infant years, one cannot deny the many unique educational 
ideas and essential services which have been introduced across the 
nation. 

Many successful Title III plans have served several districts 
simultaneously. In fact, regionalization is the major political result 
of the passage of Title III. Public and non-public schools, cultural in- 
stitutions, medical facilities, governmental units and other diverse 
agencies have combined to produce educational progress on local, 
regional and state levels. Since a union of such resources is essential 
to the success of the supplementary services center, state advisory 
councils should continue to support these cooperative ventures. 

”^"The Continuation and Strengthening of Title III, " Memoran- 
dum from the Title III National Study Team to the Commissioner of 
Education, April, 1968. 
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Universities, school districts, cities and suburbs can no longer 
go it alone. Invention is rare and must be shared. Information about 
new and exemplary instructional programs and practices should be ex- 
changed and studied by educational practitioners everywhere. The 
costs for new services, staff, building, supplies and equipment must 
be shared. As student numbers increase from a 1965 total of 54. 7 
million to a 1975 figure of 63. 3 million, local taxpayers will expect 
state and federal sources to assume a greater share of the costs of 
education.-^ Hopefully, the transition of Title UI authority to the 
states will spur legislators to devise state aid formulae, capable of 
supporting and replicating exemplary programs initiated under Title 
III, upon expiration of federal funding. Successful Title III centers 
can be state prototypes for permanently state aided cooperative ser- 
vices centers similar to New York State's Board of Cooperative Edu- 
cational Services Centers (BOCES). 

The typical state can offer a cross-section of Title III centers, 
featuring pupil personnel services; diagnostic clinics; computer sys- 
tems for the storage of research and administrative data; pre-school 
readiness and dropout prevention programs; cultural, health and voca- 
tional projects; guidance and social services for deprived children; 
curriculum development services; graphics and materials production 

—^"Projections of Educational Statistics to 1975-76," U. S. De- 
partment of Health, Education and Welfare, Office of Education, 1966. 




agencies; teacher training programs dealing with new instructional 
strategies; human relations and the observation and analysis of learn- 
ing and teaching; and outdoor education programs.. 

Prior to the adminstration of Title III, these programs were 
frequently offered on a fragmented basis by local school systems, or 
else were totally ignored. The need for such services was undeniable, 
but money and courage were lacking. Title III provided ’ thinking 
money, " as well as a national psychological commitment to will, 
courage, and creativity.-^ The educational "oddball" found a friend 
in Title III. Boards of Education and superintendents of schools un- 
willing to commit local funds to provocative but unproven ideas, and 
fearful of risking personal and collective reputations on "unique" pro- 
posals, found renewed courage in the passage of PL 89-10. New pro- 
grams popped for the physically and emotionally handicapped, infor- 
mation retrieval and data processing, in-service training, film libra- 
ries, vocational courses, etc.; not truly inventive perhaps, but new 

and necessary to the sponsoring agencies. 

Why were such services suppressed for so long? Title III pro- 
posal writers indicated time and af^in the inability of the applying 
agency (local or regional) to fund the projected activity under existing 
financial limitations. Other realities cited by project directors in- 
cluded the lack of fiscal independence, taxing limits, disinterested 

—^National Study Team to the Commissioner, loc. ciL 
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stcite legislatures, provisional boards of education, inadequate state 
aid and lack of money for research and development. 

Advisory Councils should know that supplementary service 
centers need not be the products of earthbound thinking. Title III 
centers should be designed: 

1. To serve as urban communications centers , incorpora - 
ting audio and video program banks available through dial retrieval 
people living throughout the city . Pre-selected video and audiotaped 
programs could be packaged and transmitted to display stations in- 
stalled in tenement lobbies, supermarkets, laundromats, schools, 
libraries, hospitals and universities. These programs could contain 
information ranging through such topics as employment and housing 
opportunities, transportation instructions, legal procedures and medi- 
cal information. For school purposes, direct skill teaching tapes in 
typing, speech therapy, musical and athletic skills, sculpture, public 
speaking, etc. could be offered. The potential is phenomenal, and 
the hardware is available. High transmission costs and lack of perti- 
nent software are formidable obstacles to the immediate success of 
such electronic networks. Nevertheless, it seems incumbent upon 
state leadership groups working under Title III to encourage solutions 
to such problems through experimentation with regional audio-video 
information retrieval networks. 

2. To extend beyond the borders of "education . " New 
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instructional techniques and technology should be employed to convey 
information and service in the areas of health, welfare, law, trans- 
portation, housing and employment. Community oriented job centers, 
housing offices, medical clinics, legal aid societies, facilities for 
performing arts should be housed alongside school-oriented, audio- 
video program banks, handicapped children’s programs, in-service 
teacher training centers, film libraries, data retrieval centers, 
graphics and publications services, and modest television production 
facilities for the preparation of special videotapes to be transmitted to 
the community as new needs arise. 

3 . To incorporate ^ combination of service , research , and 
planning programs. Title III centers should be more than hardware 
exhibits catering to a now-and-then clientele of curiosity seekers. The 
permanency of supplementary services centers will be assured when 
the functions such centers perform are considered paramount to the 
forms they take. Services (graphics, publications, etc.) create a 
bond of identity between the center and its users. At the same time, 
research and planning must occur so that intelligent decisions to ter- 
minate, continue or expand on-going projects can be made. The Title 
III National Study Panel has contributed several papers on evaluation 
which should be studied by State Advisory Councils. It is important 
that state agencies and advisory councils achieve literacy in certain 
new trends in the field evaluation of Title III projects. Reports by 
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Guba, Stufflebeam, and Stake describing evaluation models for Title 
III field projects should become required reading for teachers, ad- 
ministrators, consultants, state department and other project per- 



sonnel. 



1 1 



Advisory Councils will be hard-pressed to make politically 
strategic" Title III allocations throughout each state. Proposals for 
supplementary centers should be distributed to disinterested, out-of- 
state content consultants in order to ensure objective, impartial ap- 



praisals. The involvement of such consultants would be insurance 
against a Title III statewide plan which might fail to give priority to 
the urgent needs of the cities or to the local educational agency pro- 
posing a unique and creative program, but lacking the initial protec- 



tion of a regional umbrella. 

It is imperative that state authorities not permit Title III cen- 
ters to degenerate into carbon copies of established programs. It is 
proper to transfer and apply good ideas adjusting for local variables. 



1 /. 



Miller, R. I. , ed. , "Notes and Working Papers Concerning 
the Administration of Programs Authorized Under Title III of Public 
Law 89-10, The Elementary and Secondary Education Act of 1965," 
Subcommittee on Education, United States Senate, April, 1967, pp. 
305-320. A second study prepared by the Title III National Study Ef- 
fectiveness of Evaluation Sections in Approved PACE Projects with 
Recommendations for Improvement," February, 1968, contains perti- 
nent material by Guba (Chapter V, p. A- 203), and Stake (Chapter V, 
p. A- 2 18). Stufflebeam' s model for the evaluation of Title III proposals 
is contained in his paper, "The Use and Abuse of Evaluation in Title 
III," Ohio State University, Evaluation Center , Columbus, Ohio, July, 
1967. 
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It is also proper to attempt to improve the condition of education 
through regional sharing of high quality services. Nevertheless, it 
would be contradictory and wasteful to violate the original spirit of 



Title III by failing to encourage, identify, and fund educational inven- 



tion and creativity, wherever it is found, 



















SCHOOL-COMMUNITY RELATIONS 



by John W. 
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Recent events in New York as well as in other comm. unities 
throughout the country have highlighted the importance of developing 
more effective school-community relationships. Typical patterns 
which have developed over the years are not adeouate for the stormy 
present. The urgency of discovering new and better ways of assuring 
that the school and the community it serves move together toward 
the accomplishment of common goals is attested by the tragic events 
which are so much a part of the current school story. It would be 
incorrect to assume that spot news accounts present the full picture 
because to a large degree it is still a case of "man bites dog. " It 
would be equally shortsighted, however, to fail to recognize the danger 
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signals as they appear. 

If used to its maximum potential, ESEA Title III provides a way 
to accomplish improved school-community relationships. There is 

— ^ Superintendent of Schools, Atlanta Public Schools, Atlanta, 
Georgia. 
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considerable evidence to indicate that this full potential has not yet 
been realized, but there is also evidence that progress has been, and 

is being, made. 

The following comments are related to a review of 30 selected 
projects illustrative of efforts directed toward the expansion and/or 
improvement of school- community relationships. Broad generaliza- 
tions are frequently in error as they apply to specific cases, and the 
evaluation of PACE projects is no exception. Outstanding exceptions 
to the following generalizations can be found among the 30 projects re- 
viewed. This is further evidence that Title III, if its potential is fully 
realized, can make a vital contribution in the area of school- community 

relationships . 

1. Many projects were aimed at the development of additional 
services, obviously needed and excellent in themselves. In too m^ 
instances , however , these additionaj servic e were considere d,^ 
integral part of the ongoing school program. It often appeared as 
though the school intended to continue its program as usual with the 
hope that deficiencies would be taken care of through the special pro- 
ject without "interrupting" the normal school operation. 

2. Only a few projects were directed toward the discovery 
and implementation of new patterns of school- community relationships, 
although a number had the potential for doing so. There appeared to 
be more interest in doing things for the community than with the 
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community, and the potential for establishing new patterns of school- 
community relationships was more incidental than by specific design. 

3. Most, if not all. Title III projects could appropriately in- 
clude a community involvement component which would contribute to 
improved school- community relationships — not from the standpoint of 
merely providing information about the project, but for the specific 
purpose of improving results through effective utilization of community 

human resources. 

4. Many projects appeared to be designed in a manner 
which would permit their discontinuance without serious consequences. 
This is understandable in view of the fact that many local budgets 
would be unable to abso^rb project costs. It is nevertheless true that 
needed improvements in school- community relationships cannot be 
accomplished by special projects alone. Such improvements must 
come through a total school commitment, and special projects should 

be a part of the total undertaking. 

5. The financial shock resulting from the expiration of a 
Title III grant could be cushioned somewhat through the design of pro- 
jects which would incorporate funds from several sources. This ap- 
proach would also contribute to the desirable goal of making the pro- 
ject an integral part of the ongoing school program in a way that 
would help assure maximum impact. Lasting improvement results 
not so much from the addition of "separate” school- community 
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activities as from an expanded concept of the role of the school in this 
area and from the increased competence of the total staff. 

6. Several school- community projects were directed toward 
an expanded school program utilizing school personnel and physical 
facilities in the afternoon, evening, and throughout the summer for 
both school age pupils and adults. There is an urgent need for more 
developments in this direction. Also, projects of this type point the 
way and set the stage for realistic community involvement in the total 
program of the school. 

Current questions related to community control of schools 
as opposed to community involvement would also be answered, at least 
in part, as the school moved toward becoming the kind of institution 
that was more concerned about the needs of the total community. This 
goal can be achieved in many school systems through proper utiliza- 
tion of existing federal resources, including Title III. If this need 
were given special emphasis. Title III could stimulate developments 
which would expand the role of the school and contribute substantially 
to improved school- community relationships. Several excellent pro- 
jects demonstrate this fact. 

7. Vocational education offers many possibilities for de- 
veloping improved school- community relationships. In general, how- 
ever, few Title III projects have been directed toward achieving an 
improved vocational program. This fact is further evidence that 
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\ there now exists a separateness— vocational education, on the one 

hand, and general or academic education, on the other which should 
be eliminated. A forward step in this direction would be to finance a 
part of the vocational program from sources other than vocational 
funds. Title III should not overlook the needs in the vocational area 
and should not fail to utilize the techniques for community involvement 
that vocational education has developed. Many urban problems will 
respond to the County Agent-Home Demonstration Agent approach. 

Local - State - Federal Relationships 
Congressional action transferring the administration of most of 
the Title III program to the states has far reaching implications for the 
future of the program. The action has both advantages and disadvan- 

I tages: 

f 1. In the drive toward educational change and improvement, 

little is accomplished by attempting to circumvent what frequently has 
been referred to as the "establishment. " If the establishment is a 
roadblock which restricts or prevents desirable change, it is still 
more fruitful to work to change the establishment than to try to cir- 
cumvent it. 

; 

S E. All state departments of education are not equipped to 

exercise the dynamic leadership that is urgently needed to continue 

I 

i the forward thrust that Title III has thus far stimulated. Because of 
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this fact many individuals may view the change in administration of 
Title III as a tragic mistake. If viewed in proper perspective, how- 
ever, a possible short-range mistake can become a long-range ad- 
vantage. 

The structure of public education is such that few funda- 
mental, lasting improvements can be generally accepted and put into 
operation without the support of state education officials. At present 
this is a moot questionbecause the change in administration has been 
made, and unless the structure is changed again, which is doubtful, 
there is no alternative but to work within the congressional decision 
as it now stands. This does not release the U. S. Office of Education 
of its responsibility for leadership. As a matter of fact, this respon- 
sibility is iixcreased and emphasizes again the urgent importance for 
all concerned to understand the dimensions of the term. Leadership 
and authority are not synonymous and frequently are not even com- 
patible. Leadership can; and must, be exercised at the federal level 
to assure the continuation of high quality Title III programs and the ul- 
timate accomplishment of the far reaching purposes for which the 
Title was established in the first place. Leadership, especially at the 
federal level, is best exercised when it is not confused with authority. 

3. Under state administration there will be a natural tend- 
ency to spread Title III funds in a manner which assures participation 
by all systems in the state. This approach will likely dull the "cutting 
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edge" that Title III was established to stimulate, but in the long range 
view may also have some advantages. What is the cutting edge for 
one system may be an outmoded approach for another. Although it 
should never be necessary to discover the wheel all over again, it is 
also true that a school system improves by moving from where it is. 
Certainly, efforts should be made at least to. stimulate a start on the 
part of those systems that have so far to go. Spreading of Title III 
resources can contribute to the accomplishment of this purpose. 

4. Cooperative efforts involving a group of school systems 
banded together for the better accomplishment of common objectives 
represent some of the most successful Title III projects. State ef- 
forts to spread Title m resources in a manner to involve all systems 
will likely stimulate the development of more cooperative projects. 

This will be moving in the right direction and might ultimately result 
in desirable changes in district lines. 

Comments - Recommendations 

1. Flexible funding periods should be established f^ Ij^le, 
III projects . It is not consistent with the purpose of Title III to assume 
that all projects should continue in operation for the same length of 
time. The funding period should be related to the purposes to be 
achieved, which might be more or less than three years. 

2 . Project proposals which utilize funds from several 
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sources should be encouraged . The importance of this approach is not 
reduced under state administration and should be facilitated through the 
cooperative development of guidelines. 

3 . Restrictions on the use of Title III funds for building con - 
struction should be removed . Obviously a proper balance should be 
maintained between construction and program, but it is unrealistic to 
assume that there is no relationship. Also, it is most unlikely that a 
project involving construction will be abandoned at the expiration of 
the project period. 

4. , Packaging (projects utilizing funds from several sources) 
should not be left entirely to local initiative . Within present legal re- 
quirements it should be possible to combine categories, sources of 
funds, and guidelines to encourage broader approaches to the solution 
of educational problems. Unless encouraged to do otherwise, many 
states and local systems will interpret guidelines in terms of the letter 
rather than the spirit of a regulation. Many individuals think it is un- 
realistic to hope that leadership in this direction could come from the 
federal level. There is considerable evidence to indicate that the 
spirit of the law is frequently violated at state and local levels. Of all 
the federal programs directed toward the improvement of education, 
Title III is probably the most appropriate one to exercise leadership 
in this area. 

Title III funds administered at the federal level can be 
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effectively utilized to preserve current efforts ^ encourage the d^- 
velopment of exemplary programs . Continuation of such programs 
could then move to financing through state administered funds. It is 
believed that such efforts would also contribute to the development of 

the right kind of federal- state relationships. 

6- A high priority should be assigned to projects designed 
to prepare teachers to cope with situations resulting when scho^ol- 
community value systems are different from their own . Title III- 



EPDA might appropriately join forces here. 



PRIORITIES FOR PACE 



*/ 



by Robert J. Havighurst“ 



This memorandum is based on the proposition that each state, in 
the future, will have from $500, 000 to $ 10 million from the Federal 
government to use for the improvement of its educational system through 
experimentation, demonstration and evaluation of new practices . The 
amount for each state will depend primarily on its population, though 
it may quality for supplementary funds if it establishes and maintains 

an excellent program. 

Four general types of programs or projects may be supported 
by PACE funds: 

1. New and highly experimental projects, carried out by 

creative people on a pilot-project basis. 

2. Demonstration and dissemination of new practices that 

have been tested and proven. 

3. Supplementary centers with programs that serve a num- 
ber of public and private schools in a community. 



—^Professor of Education, School of Education, University of 
Chicago. 



- 126 - 



o 

ERIC 















.UUi^Ur-UWMIU 




-127- 




4. Cooperative programs which pool the resources of 
leveral school systems to meet needs that they are too small to cope 



with otherwise. 

The amount of money involved will not be great, when compared 
with the total state budget for education. This money should be kept 
fluid and should never be tied up for more than two or three years on 
a given project without a thorough evaluation and deliberate decision 
on the advisability of continuation for a second and final two or three- 
year period. A great danger is that the money in the fund may be com- 
mitted to a set of reasonably good programs which cannot be funded by 
the regular budget of the state, or by the local community. In this 
case, these programs might become quasi-permanent, standing in 
the way of the main function of PACE, which is to try out and evaluate 



innovations. 

In order to maintain fluidity and efficiency of use of the fund, 
there might be a small State Commission on Educational Development, 
consisting of leading citizens, with an executive secretary full or 
part-time — who is not in the State Department of Education. He might 
be a faculty member from one of the colleges in the state. This Com- 
mission might issue annual reports as well as news releases at 



auspicious times. 

Whether this State Commission should have the power to re- 



ceive 



proposals and allocate the funds is not clear to me. Perhaps it 
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should have this power, though there might be considerable resistance 



"1 



from the State Department of Education. It might be better for the 
State Department to set up its own machinery for allocation of the 
funds, and to regard the State Commission as a friendly but indepen- 
dent observer and judge of its performance. 

Public information . This kind of agency would need to do a 

skillful job of reporting to the public on PACE projects and on the 
PACE procedure for stimulating creative innovation. The support of 
the public would be needed for the continuance and extension of innova 
tive projects. Therefore, this agency would need to establish in the 



state an atmosphere favorable to innovation, and favorable to the sup 



port of innovations that have proved valuable and viable in the school 
system. 



Sugge stions for the State Program 
Each of the four types or categories of projects referred to 
above might be adapted to conditions in the state. Perhaps the follow 

ing suggestions would be useful. 

1. New , experimental projects . These projects should 

generally be small in scope, carried on by individuals or teams of 
people who can demonstrate both a drive for innovation, and compe- 
tence in the area they choose to develop. Some preference should be 
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given to younger people, partly because they could not easily secure 

1 

support elsewhere. 

The local school system might be asked to supply some 
support for the project in those cases where the^cost is fairly large 
and the school system has a large budget. At the very least, the 
superintendent should write a letter of support for the project, and he 
should indicate that the person or persons in the project will be given 
as much informal support and assistance as possible. 

An area of special importance for the future iSthat of 
the arts and humanities. New curriculum materials and new courses 
in this area might be given some priority. To date, it seems that 
most of the PACE projects in this area have been concerned with 
bringing the school system closer to community cultural agencies, 
such as museums and theaters and orchestras. While this is desirable, 
there is a growing need for increased attention to the arts and humani- 
ties in the regular school program. 

2. Demonstration and dissemination of new practices . 

There is an art to dissemination that is not well understood. It prob- 
ably recjuires a combination of administrative support, in-service 
teacher training and t^e provision of new instructional materials. 

Perhaps the State Department could employ one or more 
consultants on the dissemination of new practices who could work with 
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local school systems. The State University might provide such a per- 
son as a service to school systems. 

It should be clear that dissemination does not necessarily 

follow experimentation in a given school system. Often, a new prac- 
tice which has been developed and tested elsewhere will be adopted by 
a school system — such as a non- graded primary school, team teaching, 
or foreign language in the elementary school. The adoption and imple- 
mentation of this new practice in a system may require special help 
over a short period of time, help which can probably be paid for from 

PACE funds. 

3. Supplementary Centers that serve a number ^ school_s_ . 
The establishment of a supplementary educational center to serve a 
number of schools is an expensive operation which can seldom be com- 
pletely paid for out of PACE funds. Yet this form of program may be 
very important as a means of promoting racial or economic integra- 
tion of children, and as a means of bringing private schools into co- 
operation with public schools. 

The supplementary center idea is so important that it 

might be given special consideration by a particular school system as 
a project for which funds would be collected from several sources. 
Certain <^federal funds might also be available, especially Title I funds 
under the Elementary and Secondary Education Act. 

In a medium- sized city of from 50, 000 to 100, 000, a 
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supplementary center could probably be installed in a partially used 
building, and then supported by a variety of sources, with the intention 
of making it a regular part of the budget as soon as possible. 

4. Cooperative projects in rural and sparsely settled areas. 
A large number of small cities and rural counties lack the resources 
for such valuable educational services as a vocational high school, a 
library, a visual aid center and classes for handicapped children. 
There is now a general move toward informal cooperation among such 
communities, or toward an intermediate regional school district which 

would provide such services. 

PACE funds could not fully support many such programs, 

but they might be used to start one or two as models. Here, the State 
Department of Education should provide stimulation and leadership. 
Some sort of Office of Educational Innovation might be maintained by 
the State Department of Education, perhaps in cooperation with the 
State University. 

Cooperation Among State s 

If federal funds are to be used effectively for innovation, but 
continue to be distributed through the states, at least one-half of the 
states will have to improve substantially their State Departments of 
Education as agencies to stimulate innovation. One way to bring this 
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about would be through cooperation between the various State Depart- 
ments of Education. 

The Compact of the States might create an Association for Edu- 
cational Innovation, made up of State Departments in the less urbanized 
states. This Association might maintain a small staff of consultants, 
and might hold conferences on the different aspects of educational in- 
novation. 
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PACE AS PROCESS 



.*/ 

by Norman D. Kurlancf" 



The focus of this paper will be on the impact of the PACE pro- 
gram upon educational planning and activity at state and local levels. 

It is designed to indicate some of the possible ways in which PACE as 
a whole has affected certain key educational processes, and to suggest 
hypotheses for an evaluation of the entire program. 



PACE is Proposals 

The most obvious new activity associated with Title III has been 
proposal writing. Never before have so many public school people 
worked so long and so hard on project applications. The 6720 propo- 
sals submitted by public schools in the first three years of Title lU 
probably exceed by far the total number of proposals to private founda- 
tions prepared in the past by public schools. A whole new speciality of 
proposal design and writing has developed, and thousands of school 



—^Director, Center on Innovation, New York State Education 
Department. 
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people and others have gone through the exercise of preparing propos- 
sals. V\That effect has this had on them, and on American education 
generally? Has the process of proposal preparation itself had any 
positive effects, or is it of value only as a mechanism foi distributing 
large amounts of federal funds? To substantiate the conclusion that 
the process has been valuable, evidence needs to be gathered to deter- 
mine whether or not certain kinds of changes have occurred that can 
be reasonably attributed to the process. Among the changes which 
should be looked for in the behavior of educators are the following: 

1. A greater understanding of what is meant by educational 

needs, and how to assess them, has been developed. 

2. The ability to write precise educational objectives has 
been increased, along with the ability to detail specific procedures 

for achieving those objectives. 

3. The ability to design evaluation procedures to test the 

achievement of the objectives has improved. 

4. The technique of relating budgets to programs has been 

mastered. 

5. People and agencies have learned to plan and work to- 
gether to develop programs of mutual interest. This includes school 
districts working with each other, with non- public schools, with insti 
tutions of higher education, and with other cultural, educational and 
community agencies. 
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6. A significant number of rejected proposals have neverthe- 



less resulted in undertaking the proposed activity because of the inter- 



est and commitment generated during the proposal preparation. 

These are hypotheses which can be empirically tested, both by 

t 

comparing PACE proposals of 1968 with those of 1966, and by studying 
the behavior of educators when dealing with other programs. Has 



there been a transfer of learning from the Title III program to other 



activities? Do American educators who have had the PACE experience 



now do more careful planning of other programs, particularly those 



supported by local funds? Do they pay more attention now to the 
assessment of need, the specification of objectives, the design of 



procedures, and to evaluation? Have they continued with other agen- 
cies, working relationships that were initiated by Title III? 



The answers to these questions should be included in the over- 



all evaluation of Title III. If Title III has produced a major change in 



the behavior of American educators, this could rank equally with the 



value of the projects themselves 



If, on the other hand, there has been little learning and little 



change of the kind described, and if the preparation of proposals has 
been a narrow technical task with little generalizability, then serious 



questions must be raised about the value of the proposal approach. 



For it must be remembered that this is a costly, time-consuming. 
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and often frustrating process, as all those who have worked on propo- 



sals know. 



While it is impossible to determine what it costs to develop and 



process a proposal, if one takes into account the time of all those in- 



volved, including school administrators, advisory councils, readers 



and reviewers at the State and Federal levels, an average of $10, 000 



a proposal would probably be a conservative estimate. This means 



that the 2639 proposals approved to date represent an investment of 



over $26 million. This cost must be added to the cost of the projects 



themselves to get a fair estimate of the total cost of Title III. Is it 



worth it? 



Since we are not likely, for awhile, to abandon the proposal 



approach, no matter what the evaluation may show, a better question 



would be: Can we do more to ensure that the proposal process itself 



will have educational value? Now that the administration of Title III 



is being transferred to the states, they have an opportunity to examine 



the process and to devise procedures which will increase the prob> 



ability that the objectives listed above as hypotheses will, in fact, be 



realized. Whether or not they are realized can then also be evaluated. 



thus making the Title III program itself exemplify the very process it 



is seeking to promote among the applicants. 
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others for any Title III funds it gets, and competition is intense. 

Since the beginning of PACE over 6720 applications have been received, 
and only 39 percent of them have been funded. Applicants have known 
these odds, yet they continue to apply, facing the high risk of frustra- 
tion as well as the high cost of proposal development. 

Is this competition desirable, or would it be better to move to- 
ward the ESEA Title I model where each district has an allotment and 
only needs approval of the proposal in order to obtain it? A definitive 
answer should be given only after a more extensive evaluation of 
PACE. Some of the lines along which further investigations might 

proceed can be suggested. 

One test would be to look at the answers to the questions posed 
in the first section of this paper. If significant learning takes place 
not only among the successful applicants, but among the unsuccessful 
as well, then, PACE may actually be a relatively inexpensive, if not 

entirely painless, form of in-service training. 

This still leaves the question of whether or not the competition 
itself is beneficial. The test of the hypothesis that it is beneficial 
would be, in part, a comparison of the quality of proposals submitted 
in the early rounds with those submitted more recently. If proposals 
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are improving (and people who have read them from the beginning have 
the impression that they are), then at least this shows that people have 
learned to write better proposals and are spurred to this effort by the 
competition. 

A better test would be whether the projects implemented now 
are better than those started earlier. Have people learned not only 
how to write proposals, but how to design and implement programs? 

In addition, the study should include a comparison of earlier and later 
Title I proposals. If there are similar improvements, then competi- 

tion may not bo necessary. 

A more significant test of the value of competition would be 
whether or not both professionals and laymen now tend to judge a 
school system partially in terms of the number and quality of the in- 
novative programs it has. Clearly, this ca:n be a superficial and even 
dangerous test if attention is paid only to the number of innovations, 
rather than to their character, quality, significance and endurance. 

The emphasis must be not on innovation for the sake of innovation, 
but on innovation for the sake of improvement. Competition in Title 
III can be judged beneficial if it can be shown that there is increasing 
awareness that a changing society requires change in education as well 
as increasing demand for quality and evidence of improved performance 
in the innovations which have been tried and adopted. 
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PACE is Evaluation 

Of course, the critical issue is: can we tell which innovations 

really are improvements over traditional practice? It is to this issue 
that this third element of the PACE process is addressed— the em- 
phasis on evaluation. This may be the area in which PACE and other 
ESEA programs have had the greatest impact on American educators. 

In the past three years, educators have had to face a demand 
for accountability to an extent never known before. Evaluation re- 
quirements were written into ESEA by Congress, and each year at 
appropriation renewal time. Congressmen have asked what the evalua- 
tions show. No one can as yet be satisfied by the evidence that has 
been developed to date, but clearly, the ability of education to evaluate 
its programs is improving, and the formerly hostile attitudes of edu- 
cators to evaluation are beginning to change. Evaluation is being 
accepted, and effort is being put into improving its effectiveness. 

It will still be several years yet before we can say with confi- 
dence that evaluation has become an integral part of the process of 
education in this country. If it does become an integral part of the 
process, much of the credit can certainly be given to PACE. 



PACE is F ederal Phase - Out 

A central feature of PACE has been its treatment as ’’seed 
money, ” to be used for developing promising ideas and for 
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demonstrating those that prove effective. It has not been viewed as a 
source of sustained federal support for individual programs. 

The *'phase-out" requirement has forced school districts to do 

9. number of things they had not previously done: 

First, it has forced them to begin using evaluation data in mak- 
ing decisions. When a decision must be made on whether or not to 
continue an activity with local funds, it is hard to avoid using the 
available evaluation data. This converts evaluation from a ritualistic 

ornament to a practical decision tool. 

Second, phase-out policy has made districts consider, at the 

time they initially apply for a grant, whether or not they are willing 
to commit themselves to continuation if the project is successful. It 
is easy to allow the federal government to support a project at no cost 
to the district. Few superintendents or Boards of Education would re- 
fuse to submit a proposal if someone on the staff were willing to write 
it. It is quite another matter if the district must make a commitment 
to continuation, should the project succeed. Then, more serious at- 
tention is paid to the importance of the problem to which the project 
is addressed, and to the question whether or not it meets a need on 

which the district wants to spend ij^ money. 

Third, the phase-out policy has made districts consider ways 
of financing activities on an ongoing basis. One of the vital matters 
for future study will be to determine how many projects are continued 
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by the original grantee, how many are adopted by others as a result of 
the experience of grantees, and what the sources of funds are for both 
the continuations and adoptions. Do districts ’’find” new money to sup- 
port new activities, or do they shift funds from less effective, or less 

significant, activities to the new? 

The latter is by far the more significant kind of change to look 
for. It is relatively easy to add on activities, particularly when money 
can be ’’found” for them. It is much harder to substitute new practices 
for old. This requires change in priorities and change in behavior, 
both changes difficult to bring about. In the long run, however, if in- 
novation is to produce significant improvement, it must be along the 
latter path, rather than the former. There will never be enough money 
or people to meet the expanding educational requirements of our 
society. The only hope is to replace less effective practice with more 
effective practice. That means creating the more effective practice, 
proving that it _i^ more effective, and convincing those who have the 
power of decision to change for the better. These are the three key 
elements of PACE. 

^ PACE will be State Plans 

A fifth aspect of PACE which may have a profound impact on 
American education is the recent requirement for the development of 
State plans. For the first time in many states, the state education 
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agency is being required to look at the entire educational program of 
the State in a comprehensive way. It is being asked to assess the edu- 
cational needs of the State, to place priorities on these needs, and to 
develop a long-range strategy for meeting these needs. It must de- 
velop procedures for assessing proposals; it must create evaluation 
and dissemination capabilities; and it must make judgments about the 
proposals submitted to it. 

If states use well the opportunity created by the transfer of 
Title III to them, this could do as much, or more, to strengthen state 
departments than Title V. Analysis of the initial State Plans (submit- 
ted by July 1, 1968) can provide excellent base-line data against which 
future change can be measured. One of the major responsibilities of 
the USOE should be to assess the plans, and then to provide for an 
annual assessment, both of the up-dated plans and of the performance 

of the states in carrying out those plans. 

If the states prove that they can achieve the PACE objectives, 

and if the realization of those objectives themselves proves to be 
worthwhile, this will provide Congress with a channel ready for the 
flow of federal funds into education, when funds in large quantity 
again become available. 

This suggests a major reason for careful attention to the value 
of PACE as process. Federal money will, for long, be the smaller 
part of the support of education. Congress will certainly want to 
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maximize the impact of its contribution. It will certainly be impressed 
if PACE projects prove to be exciting and effective. It will be even 
more impressed if it can be shown that the very process by which its 
funds are distributed has itself had a profound effect on the way the 
other 80 or 90 percent of the funds spent on education are used. 
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EVALUATION GUIDELINES AT THE STATE LEVEL 



*/ 

by Egon G. Guba““ 

Beginning with the current fiscal year, the major control and 
responsibility for the administration of ESEA Title III funds passed to 
the several states. Each state has prepared policy statements and 
guidelines for use in the state by persons and agencies contemplating 
the submission of Title III project proposals. Since the federal law 
re(]uires that each project shall be evaluated, the states must neces 
sarily inc lude intheir guidelines some statement relevant to evaluation. 

An attempt will be made in this paper to suggest certain ele- 
ments which, in the view of the author, are manda tory for inclusion in 
any such evaluation guidelines.” The paper itself should not be view- 
ed as a guideline statement; it is not intended as a model of what such 

—'Director, National Institute for the Study of Educational 
Change, Indiana University. 

— ^Most of the ideas in this paper were prepared by the author 
for the document. Planning and Implementing Title III Evaluations, a 
statement by Robert L. Hammond, Daniel L. Stufflebeam, and Egon 
G. Guba under contract with the Division of State Agency Cooperation, 
USOE. 
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a statement should be. Undoubtedly there are many elements which 
individual state guideline writers will wish to consider for inclusion. 

It is the contention of this author, however, that a failure to include 
discussion of the elements treated here will be an oversight of dev- 
astating proportions to the public school personnel attempting to do 
a good job of designing an evaluation program. These elements are: 

(1) a definition of evaluation, (2) a statement of the criteria which 
will be applied by the state in determining the acceptability of an 
evaluation statement in a proposal, (3) a discussion of the relation - 
ship of evaluation procedures to research procedures , (4) a discus- 
sion of the aggregation problem, i. e. , how the state intends to aggre- 
gate data from the several projects in order to formulate its own 
overall evaluation report, (5) some suggestions to the local projects 
on how to provide for staffing and budgeting , and (6) overall policies. 
Each of these areas will be very briefly treated below, mainly with the 
intention of exposing the nature of the problems, rather than of making 
suggestions as to how to cope with them. 

Definition 

The term "evaluation" is used in so many different ways that a 
definition of the way in which the state may use this term is indispen- 
sible. Illustrating the variety of definitions in current use is a simple 
matter: 
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The Dictionary of Mathematics defined evaluation as 
«'the finding of a numerical value for some expression. ” 

The Dictionary of Education gives, as the first meaning ; 

of evaluation, "the process of ascertaining or judging 
the value or amount of something by careful appraisal. " 

J. B. Carroll in his book. Learning and the Educational 
Process , defines evaluation as "a process of determin- 
ing which objectives a given curriculum can attain, 
under what conditions, and for what kinds of pupils. 



• The Encyclopedia of Educational Research indicates that 
the process of evaluating "involves three distinct aspects, 
consisting of 1) selecting the attributes that are impor- 
tant for judging the worth of the specimen to be evaluated, 
2) developing and applying the procedures that will de- 
scribe these attributes truly and accurately, and 3) syn- 
thesizing the evidence yielded by these procedures into a 
final judgment of worth. " 

• ^4ichael Scriven in the monograph. The Methodology of 

Evaluation , indicates that the goal of evaluation is the 
"estimation of value," and describes two roles for evalua- 
tion: formative- — evaluation of ongoing processes to im- 

prove them, and summative — the evaluation of outcomes. 

• Cronbach, in his paper, "Evaluation for Course Improve- 
ment, " suggests that "evaluation is the collection and use 
of information to make decisions about an educational pro- 
gram. " 

• Stufflebeam has variously suggested that evaluation is the 

process of providing information for .use in decision mak- 
ing. He also identifies four classes of evaluation: con- 

text, input, process, and product. 

This listing is sufficient to give the flavor of the many different 
kinds of definitions which are extant, and which may be called^ hi t]^ 
mind of the reader when the term is use^. Some of these definitions 
come close to equating evaluation with measurement. Some equate it 
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with the process of determining cause- effect relationships. Some sug- 
gest that evaluation relates performance to objectives. Others suggest 
that value judgments are involved, thereby also implying the existence 
of criteria (values) . Others relate evaluation to decision-making. Still 
others suggest that evaluation comes in a variety of types or classes, 
each presumably with special applications. Thus, if the state says, 
"Evaluate!" without giving some indication of what is meant by that in- 
junction, many options are left to the local project designer as to what 
to do. And, since he is probably not aware himself of the many possi- 
ble meanings that could be attached to the term, he will simply do what 
his definition suggests. If his design does not meet expectations, or if 
the individual designs of the several projects in the state do not yield 
data which can be aggregated in a meaningful way because they rely on 
different definitions, who is actually to blame in the absence of a clear 
definition in the guidelines? 



Criteria 

Evaluation is itself subject to evaluation. When a particular 
evaluation design is proposed, or when a particular evaluation is im 
plemented, how can one tell whether or not it is a good evaluation? 
What criteria might the state reviewers apply to determine if they 
should approve an evaluation proposal? What criteria might the 
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individaal project staff apply to its own evaluation design or procedures 

to determine whether or not they are good? 

Undoubtedly, many different criteria could be proposed, but the 
following seem to the author to be minimal. They are divided into two 
classes. The first class arises because evaluative information is sub- 
ject to the same rules which govern the value of any information, i. e. , 
scientific criteria. The first four criteria are of this sort. But be- 
cause the information dealt with here is also evaluative information, 
it should, apparently, satisfy certain additional criteria of relevance 
to the evaluative situation. The last six of the criteria listed below 
are of this sort. 

A. Scientific criteria 



Evaluative information must meet the usual criteria posed for 



any scientific data-: 

1. Internal validity. Information is a symbolic way of rep- 



resenting phenomena. This representation must have fidelity, that is, 
there must be a good, if not a one-to-one, correspondence between 
the symbols and the phenomena. Or, to put the case in more basic 
terms, the information must be true . The status or relationships de- 



picted must hold, at least for the sample being described, if the in- 



formation is to have internal validity. 



2. External validity. Information is rarely confined to the 



particular sample from which it was extracted. In general, interest 
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goes beyond that sample to a larger group, often called a population, 
to which the internally valid findings may be generalized. In these 
cases, the information must also have external validity, that is, it 
must be generalizable to the population to which one wishes to genera- 
lize. 

3. Reliability . Information which is not reproducible has 
little value since we may assume that it is generated under random 
circumstances, rather than being determined by particular causes. 

We are most familiar with this criterion in relation to tests; if the 
test on readministration does not yield scores at least similar to those 
from a first administration, can the scores be relied upon? The cri- 
terion is thus one of replicability of findings. 

4. Objectivity . This criterion refers to the publicness of 
the generated information. If the findings are capable of a wide variety 
of interpretation, they are useless, for then they mean whatever one 
wants them to mean. Certain data, e. g. , testimony, may be quite 
replicable but might not be agreed to at all by an independent obser- 
ver. Unless all persons competent to make a judgment agree on the 
judgment, the question of objectivity is at issue. 

B . Extra- scientific criteria 



The fact that evaluative information ^s evaluative allows us to 
posit certain additional, non- scientific criteria. But, their non- 
scientific nature ought not to blind us to their crucial importance. 
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If they are not met, evaluative information will not be used, resulting 
both in a waste of the resources used to get it, as well as to predis- 
pose the potential user against evaluative information in the future. 

These non- scientific criteria are: 

1. Relevance . Evaluative information must be relevant to 

the situation it was intended to illuminate. Just what constitutes rele- 
vance is, in part, a function of the definition of evaluation used. If 
evaluation is seen as the finding of a numerical value, the information 
must be relevant to the expression whose value is sought. If evalua- 
tion is seen as relating performance to objectives, the information 
must be relevant to that performance and those objectives. If evalua- 
tion is seen as a process to aid in decision-making, evaluative infor- 
mation must be relevant to the decisions concerned. In any case, ir- 
relevant information is useless and wasteful. 

2. Significance . A great deal of evaluative information can 

be gathered in any instance, but an overabundance of information may 
confuse more than it helps. An evaluation design must therefore pro- 
vide ways for culling out information of less value, highlighting that of 
greater value. Indeed, it is the exercise of his judgment about what is 
and what is not significant (not in the statistical sense), that gives the 
evaluator one of his chief claims to playing a professional role. 

3. Scope . Information may be relevant and significant but 

not have sufficient scope to be of real use. If there are four 
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alternatives involved in a decision, or four objectives to which perfor- 
mance is to be matched, or four measurements to be taken, informa- 
tion relating to only one, two, or three of these is lacking in scope. 

4 Credibility. Even the most scientifically substantiated 



evaluative information is of no use if it is not believed. The persons 



to whom evaluators report their findings, and who must act on the 



basis of those reports, are not likely to risk their judgment on the 



advice or findings of someone they do not trust. Self-reports, hearsay 



evidence, casual observation or user-testimonies are not likely to 



stir up a great deal of corlfidence. These data simply lack credibility; 



they are, in every sense of the word, incredible. 



Timeliness. The best information is of no use if it comes 



at the wrong time, or too late. Congress, in debating whether or not 



to extend Title III for another year, needs to know "how we're doing" 



when they make that decision, not three months later. The project 



director needs to know that something is going wrong while the project 



is still under way and while it can be corrected — not at the end of the 



project. The teacher who wishes to adapt her methods to individual 



differences must have data about those differences when she is planning 



her methods, not after she has implemented them. Timeliness is 



probably the most frequently violated non- scientific criterion; "too 



little, too late" is the frequent lament of the evaluator. 



6. Pervasiveness. Evaluative information is crucial to a 
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variety of audiences: teachers, administrators, project managers. 



parents, state department officials, the Congress, and the like. If 
the evaluation design does not provide channels for getting the findings 
to the revelant audience, it lacks in pervasiveness and will have an 
inadequate impact. 

Relation to Research 

A serious problem arises from equating the methodology of 
evaluation with the methodology of research. The definitive, con- 
trolled experiment is taken as paradigm for the design of evalua- 
tions. This mal- equating has resulted in a great many difficulties 
which have been described in some detail elsewhere by the author 

and by others,— and which will only be summarized here: 

Laboratory antisepsis . The control of context required 

by the experimental model (the "laboratory") allows boring in on 
specific variables whose influence is to be identified, but simultan- 



.1 



•i 

I 










eoui 



Lsly rules out the "invited interference" which is desirable in de- 
termining how anything will operate in the "real world." 

2. Intervention. Experimentation requires experimental 



arrangements, i. e. , intervention on the part of the experimenter. 



-See, for example, Egon G. Cuba, "Methodological Strate- 
gies for Educational Change, " School of Education, The Ohio State 
University, 1965 (Mimeo.); and Daniel L. Stufflebeam, "Evaluation 
as Enlightenment for Decision-Making, " the ASCD Commission for 
Assessment of Educational Outcomes, Sarasota, Florida, 1968. 




miiii 



1 







- 153 - 



This contrived situation can result in findings sharply different from 

those encountered in a real situation. 

3. Terminal availability . The general format of the experi- 
ment provides data only at the^ of the experiment. The timing of 

the information is thus often inappropriate. 

4. Restriction to single studies . It is axiomatic that sub- 
jects can be exposed to only one experiment at a time, lest the "vari- 
ances are confounded. " But the number of new treatments, all de- 
manding evaluation, that, say, a disadvantaged inner-city child might 

properly be exposed to, are multiple. 

5 ^ The " sea of homogeneity . " The careful controls de- 
manded by the experimental paradigm often cannot be met in educa- 
tion because of the inability to sample or to assign at random. The 
comparability required is usually sought by an alternative method 
matching or pairing on the variables thought to influence the results. 
Thus the experimenter will seek two classrooms in schools of equal 
size, serving culturally similar communities, with similar numbers 
of boys and girls of equal intelligence, and so forth. Having pro- 
duced such a "sea of homogeneity" by virtue of the need to approxi- 
mate comparability, how can the experimenter expect that any single 
variation (e.g. , two teaching methods) which he will impose can pro- 
duce a difference? Even it if does, for whom is that finding genera- 



lizable? 
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6. Inapplicable assumptions . Classical data analytic tech- 
niques are based on certain assumptions, such as random sampling, 
normality of distribution, and so forth, which are not typically met in 
the educational situation. While statistics are known (empirically) to 
be "robust" in regard to these assumptions (i. e. , not influenced "too 
much"), it is questionable whether or not one can have confidence in 
data derived from situations in which the assumptions are only rarely, 
or imperfectly, met. 

7. Restricted decision rules . The decision rules applicable 
in the usual experimental case are those of statistics which permit 
judgments as to the "significance" of a finding. These decision rules 
are too limited for most educational situations, in which judgments 
other than simple "go-no go" decisions are usually required. 

8. Impossibility of continuous refinement . The chief diffi- 
culty with the experimental paradigm is that it offers no possibility 
of change during the term of the experiment, yet most educational 
evaluations are aimed at dynamic situations. We want data as quickly 
as possible, not only to let us know how we are doing, but also to tell 
us what might be done to make things better. "Adjustments" in ex- 
periments are just not possible, however, for if one changes the treat 
ment, how can one know what the findings mean? 

This brief summary sufficies to illustrate the point that 
the methodology of research is undoubtedly a poor paradigm on which 
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to base the design of evaluations. It would be useful indeed were the 
several states to recognize this problem explicitly in their guidelines. 
A statement which makes it clear that evaluations will not be judged 
on the basis of their similarity to conventional experimental design 
is imperative. Local evaluators should be instructed to desist from 
further, indiscriminate use of these conventional approaches and in- 
structed, instead, to seek means for providing the information which 
. 3 / 

they need and want. 



Evaluation takes place at a variety of levels: those of the in- 
dividual student, the classroom, the school, the district, the state 
agency and the U. S. Office of Education, to mention only the most 
obvious. Evaluative information collected at one level may or may 
not be appropriate to needs at other levels. If local evaluation data 
are to serve useful purposes at these other levels, this fact must be 
made clear in any set of guidelines, so that the local evaluations may 
be appropriately designed. 

The aggregation problem has a number of facets. First, even 
if aggregation is conceived of simply as the summing up of data col- 
lected at lower levels, it is clear that this summing up cannot occur 

— I am reminded in this context of Abraham Kaplan's "Law of 
the Instrument": Give a small boy a hammer, and he will find every- 
thing he encounters needs pounding. 



Aggregation 
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unless similar data have been collected in a common format. At a 
minimum, then, the state guidelines must specify the variables about 
which it wishes information from the local project, about the format 
in which those data should be reported, and about the scheduling of 
reports. 

A second facet is that the higher echelon organization may re- 
quire data which would not normally be collected by lower level units. 
To ask, for example, how effective Title III is as a state program , is 
vastly different from asking how effective is a particular local project. 
Data generated by local projects are not likely to be responsive to this 
question unless local projects are specifically instructed as to what 
relevant data to collect and how to collect them. 

A third facet is that evaluative criteria appropriate at one level 
are inappropriate at another. We hear a great deal, for example, 
about reducing all evaluations to the level of information about how 
children achieve in schools; this suggestion is rationalized by saying 
that the education of children is the ultimate objective of schools, 
hence all programs have to demonstrate a contribution to that educa- 
tion. Thus, we find ourselves seeking achievement data to justify the 
effectiveness of Title III as a national program. This nonsense seems 
reasonable in education, but would be rejected at once in most other 
fields; thus, no one would suggest that the effectiveness of a nose 
cone on a spacecraft is judged by noting changes in astronaut behavior. 
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nor would anyone say that the design characteristics of a carburetor 
should be tested by noting changes in driver behavior. Thus, evalua- 
tive criteria appropriate to the classroom or pupil unit probably are 
not appropriate to higher level units. Local project personnel re- 
sponsible for evaluation need to be apprised of the criteria that will 
be used at the higher levels to which they report. 

Staffing and Budgeting 

The state guidelines should contain explicit information on the 
staffing and budgeting problem. An examination by the author of Title 
III proposals indicates that in almost no cases are adequate provisions 
made for funding the evaluation proposed, or for providing the man- 
power necessary to carry it out. This cavalier approach may be the 
result of the fact that local project directors do not take evaluation 
requirements too seriously, but it more likely reflects the fact that 
they are simply ignorant of what is involved. Good evaluation requires 
more than the provision of one half-time person and a little money for 
the purchase of standardized tests. 

It seems unlikely that local projects will, in general, be of suf- 
ficient size to warrant the provision in every such budget of all of the 
specialized personnel and facilities necessary to carry out a systema- 
tic evaluation. Hence, the state may wish to set up, on a regional 
basis in the state, teams of evaluators, who will work with local 
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school projects on their individual evaluation problems. The facilities 
and personnel found in such an agency as the Title 111 financed Project 
EPIC (Tucson, Arizona) may be taken as typical of what is required. 
These teams, or the agencies in which they are headquartered, may 
also wish to carry on additional functions, such as the systematic col- 
lection and processing of base- line data for all the schools they serve. 

The state agency may also wish to set up an agency or mechan- 
ism charged with the mssion of recruiting personnel to become evalua- 
tors, developing materials for their training, and actually training 
them. It is a well known fact that evaluators are almost unavailable 
on today's personnel market. The usual sources of trained personnel, 
the universities, seem to be unresponsive to this need. Most Title III 
projects find it necessary to move a teacher or administrator into the 
evaluation post provided for in the budget because of their inability to 
find a trained person. These untrained evaluators are the special prey 
of unscrupulous consultants or firms who compound their ignorance 
with their own bad advice. Note that what is suggested here is not the 
imposition of certification requirements, but the provision of genuine 
assistance to local schools in finding trained evaluators. 

Overall Policies 

The management of evaluation, like all management, requires 
reference to a set of policies. These policies should be spelled out. 
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insofar as possible, in the state guidelines. While any number of 
issues could be addressed, it seems imperative that at least the fol- 
lowing matters be considered: 

1. The necessity for ’’ outside ” evaluation . If the state finds 
the evaluation data provided by the individual project evaluation teams 
noncredible, unless verified by an ’’evaluation auditor, ” as it were, 
specific provision for such outside evaluation should be included. If 
outside evaluation is required, the local project should be given assis- 
tance in identifying a suitable outside evaluation agency to serve this 
function. 

2. The audiences for evaluation reports . The local project 
evaluator needs to know who will be privy to the data he collects. Are 
the data to be completely public, releasable, say, to the local press 
in full form? Should different kinds of reports be prepared for the 
different audiences who may have a right to the evaluation data? 

3. Feedback . The local project will be asked to provide 
data about itself in a common format with other projects. Moreover, 
it will be asked to provide special data required by higher echelons for 
their purposes. What feedback will the project obtain in turn? Will 
its own position be identified in relation to the broader data aggregated 
by the state agency? Will the data be fed back in ways that will not 
only permit the local project to assess its relative status, but also to 
refine its procedures? 
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4. Review . Who will have the authority and responsibility to 
review the project data. What decisions willbe made that might be influ- 
enced by the evaluation, e. g. , to terminate the project? 

Finale 

The reader is reminded that the purpose of this paper was not 
to provide definitive answers to the question of what should be included 
in state Title III evaluation guidelines, but to provide suggestions about 
certain problem areas that ought to be considered in drafting such a 
document. The task of developing a viable and functional set of guide- 
lines is a formidable one, particularly if, as this paper implies, so 
many issues are still open to debate, and so few of the vital re- 
sources — people, instruments, concepts, designs — are actually 
available. But, an honest recognition of these problems, even in the 
abs ence of viable solutions, is certainly better than a continued attempt 
to gloss them over, or worse, to pretend that solutions for such prob- 
lems are known, if only by the "experts. " The field of education can 
no longer afford such glib, cavalier and dishonest responses. The anx- 
ieties of many practicing evaluators will be considerably reduced, 
and the doors to further development opened wider, by a straightfor- 
ward statement from the states to the effect that new approaches are 
needed, and by a genuine effort on the part of the state agency to pro- 
vide some of the missing elements itself. 
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INSTITUTIONALIZATION OF EVALUATION 



*/ 

by James D. Finn” 

We live in an age of analysis. We also tend to synthesize and 
systematize everything we analyze in order to solve problems. These 
processes of analysis, synthesis and systemization are some of the 
power tools of our high-order scientific-technological society, aided, 
of course, by such things as computers and punched cards which sup- 
ply data about our bill- paying habits, our blood types and our penchants 
for blonds, brunettes or redheads. 

At the moment, the concern of the study group to which this 
paper is addressed is evaluation; not mere or abstract evaluation, 
but evaluation of the successes, failures, feasibilities and non- (or 
un-) feasibilities of various Title III innovative projects set up in 
school systems throughout the United States. Funds for this effort 

“'chairman. Department of Instructional Technology, and pro- 
fessor of education. University of Southern California. 

This chapter, because of its unusual nature, is carried over 
from Report No. 1. 
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are supplied by the United States Congress and the projects are ad- 
ministered by the U. S. Office of Education, monitored by the several 
State Departments of Education, and worried about by local school ad- 
ministrators. 

This interest in evaluation, in my opinion, must be seen in a 
technical- social- political context within the entire educational enter- 
prise. The age of analysis in which we live is generating an age of 
assessment in education. Thus, we have a campaign developing for a 
national assessment program (how well are the schools doing? ); 
several states are also asking and answering the same question within 
their borders.”^ Other considerations (and pressures) aside, it would 
be no surprise, therefore, to see this evaluation Zeitgeist penetrating 
the Title III program. How well are all of these Title III projects 
doing? 



Further, as technological patterns of thinking and processing 
invade the previously primitive (from a technological point of view) 
educational culture, it is inevitable that a drive for systemization 
should begin. For example, Hammond opens his paper on evaluation 
with the statement, "The need for a systematic approach to the evalua- 
tion of innovations has become one of education’s most pressing 



~^There is the little matter of the very embarrassing perfor- 
mance of Los Angeles school children on reading tests, for instance. 
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problems."”^ Analysis of a sophisticated variety must precede sys- 

temization, and such analyses can be found not only in Hammond but 

3/ 

in Clark and Cuba, Cuba, and Stufflebeam,- to refer to some very re- 
cent examples. It should be noted that all of the work cited is excep- 
tionally rigorous and highly sophisticated. It provides an excellent 
base from which to attack certain practical evaluation problems, not 
only for Title III projects but for any instructional process; this work 

is in the high technical tradition and, as such, is relatively new to 

4/ 

professional education.”" 

Further, another sign of the analysis- synthesis- system ap- 
proach to evaluation in education is the continual invention, develop- 
ment and refinement of instruments for use in evaluation processes. 



•“^Robert L. Hammond, Evaluation at the Local Level , Tuc- 
son, Arizona: Project EPIC (mimeo), undated, p. 1. 

— ^David L. Clark and Egon G. Cuba, An Examination of Po- 
tential Change Roles in Education . Bloomington, Indiana: The Na- 
tional Institute for the Study of Educational Change (mimeo), undated. 

Egon G. Guba, The Basis for Educational Improvement , 
Bloomington, Indiana: The National Institute for the Study of Educa- 

tional Change (mimeo), July, 1967. 

Daniel L. Stufflebeam, The Use and Abuse of Evaluation in 
Title III , Columbus, Ohio: The Ohio State University Evaluation Cen- 

ter (mimeo), July, 1967. 

“^It may be a sign of age, but the writer can remember when 
the word "evaluation" was used as an excuse in parts of the educa- 
tional community to avoid rigorous research; evaluation meant that 
anything went — and, in many places today, it still does. The new 
tradition will obviously change things for the better on this point. 
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Test-makers are everywhere, inventing measuring devices ranging 

5 / 

from pencil and paper tests to simulators. Cuba suggests many new 

. . 6 / 

measuring and feedback instruments are needed. 

The general objective, then, seems to be in the direction of 
systemization of evaluation procedures. Sharp analyses, increasing 
and better instrumentation, process studies are all leading to Dr. 
Hammond's "systemtic approach" to evaluation in general, and, if 
this study committee is any indication, to a systematic approach to 
evaluation for Title III projects. Such a movement to system should 

lead to a great deal of improvement. 

I would like to point out, however, that this movement toward 

system in evaluation also may not lead to improvement. For, in order 
for an evaluation system to be applied across the country, it is neces- 
sary first to institutionalize it; this is to say that, unless other means 
are invented, the evaluation system must be initiated, monitored and 
controlled by a bureaucratic system. Institutionalization of the 



5 / 

“" This phenomena can be seen among technologically oriented 
graduate students. One of my students completed a study on the evalua- 
tion of visual material by photographing and then measuring the eye 
pupil size of the evaluator and comparing it with his stated evaluation. 
Another is going to measure pulse pressure in much the same way. 

Both of these projects originated with the students themselves. It is 
no accident that Egon Cuba ([cited above,) did some very complex studies 
of television using an eye-movement television film setup a few years 
ago. 



“• Guba, op. cit . 
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evaluation process could destroy the innovative possibilities of Title 
III. 

There is nothing inevitable in this potential destruction of the 

7 / 

innovative process by the technical organization” for evaluation. 
However, if the evaluation processes as institutionalized are not to 
be made into a missile system aimed at the heart of educational inno- 
vation, additional analysis and invention is absolutely necessary. The 
remainder of this paper will examine this problem and, in addition, 

report some observations on the evaluation provisions of a number of 

1 

Title III proposals which were studied in some detail; hopefully, the 
problem examination and the proposal examination can be tied together 
to develop some recommendations to close the paper. 

“Why Evaluation Anyway ? 

The basic question that needs to be asked to begin this analysis 
is: why is eyaiuation important in the educational enterprise? There 

are at least fiye purposes or reasons that can be presented in answer 
to this question. These reasons are: (1) to add to the substantiye 

knowledge of educational processes, (2) to proyide information in 
order to adjust, discard, or otherwise change the application of an 
ongoing education process, (3) to proyide justification for a 

7/ 

I should make explicit that I belieye the eyaiuation process 
discussed aboye is a sub-technology within the broader concept of in- 
structional technology. 
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political- social- economic action relating to education, (4) to create 
a product (usually paper) which can move through educational bureau- 
cratic systems and thus keep these systems operation, and (5) to 
provide instruments which may be used to carry information on the 
success of the process to the educational community. These five pur- 
poses do not necessarily operate in a discrete fashion; in other words, 
in any one situation several may appear in the form of a mix. It is 
fairly easy, however, to identify the emphasis in each case. The five 
purposes will be briefly discussed in the paragraphs that follow.’ 

The distinction between the first and second has been noted by 
many of the recent analysts, such as Stufflebeam and Cuba. The first, 
measurements conducted under carefully controlled conditions, the- 
oretically provides materials for the corpus of educational research; 
as such, the results should add to the substantive understanding of 
educational processes. And, as has been pointed out many times, 
such results are rarely directly applicable to the problems of the 
practitioner and are of little use to decision-makers. It is possible, 
however, for such product oriented research to come into existence 
as a fall-out or byproduct of a much more comprehensive evaluation 
procedure. The use of such research techniques as the only means of 
approaching evaluation has been amply criticized in recent years. 

The second purpose for evaluation is now thought to be the 
most important when examining ongoing innovative projects in 
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education, such as those set up under Title III. Here the decision- 
maker gets information on a feedback system which tells him how well 
the process is going, what changes need to be made, etc. Various 
models of this evaluation procedure have been proposed (see footnote 

3). 

This feedback evaluation system (if it may be called that), de- 
signed to aid decision-makers dealing with practical educational prob- 
such as the operation of a Title 111 innovative center, has not 
yet been criticized to any extent due to its novelty for the field of edu- 
cation and the careful construction of the emerging theoretical models. 

However, the feedback evaluation system is open to criticism. 
It assumes, at the outset, that the decision-making process in a given 
school, school system or other educational entity is rational. It is 
not. The folklore of education is filled with examples of the school 
business manager selecting curricular materials, the high school 
dean of women throwing Salinger out of the library, and others too 
nQmerous to mention. Prior questions have to be asked whenever the 
feedback evaluation system is proposed. Who or what group is the 
decision-maker? How does the power structure really work? What 
are the motivations? Unless these questions are answered and the 
rationality or irrationality of the particular system is analyzed, the 
beautiful, precise and rational models of the feedback evaluation sys- 
tem will not work — or, at the very least, work very imperfectly. 
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The third purpose for evaluation is the purpose of justification. 
In this case, a board of education, a state legislature, a committee of 
Congress or numerous other bodies both public and private need in- 
formation in order to take some action respecting education. This ac- 
tion may be in appropriating funds, hiring additional remedial reading 
teachers, purchasing a language laboratory, etc. Or, in the opposite 
case, it may be to fire the superintendent, set the building program 
back two years or reduce the audio-visual appropriation by forty per- 
cent. These actions are justified by evaluation, whether formal or in- 
formal. In the case of the disposal of school personnel, the evaluation 
before action may be choleric and personal; increasingly, however, 
as statistics become everyday playthings of the mass media, justifi- 
cations for political- economic- social educational action are couched 
in scientific garb, whether really scientific or not. We are all familiar 
with arguments which press in opposite directions for action by some 
public body based on the same evaluative report. 

While it is obvious that public bodies with the appropriate 
authority over education have every right to evaluative information 
and, in fact, often need more than they get, it is equally true that the 
development of justification ought to be a secondary objective of edu- 
cational evaluations carried out by professionals. This, of course, is 
a value judgment. It can be argued, for example, that the effort ex- 
pended in developing a particular kind of justification evaluation in 
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order to save a program known to be good is more important in the 
real world of politics than more technically adequate professional 
evaluation. The answer to this problem, it seems to me, is to pay 
attention to the two elements in the old cliche about the tail and the 
dog. An evaluation program set up only for justification purposes is 
unprofessional; a professional evaluation procedure on a program 
that is demonstrably good ought to develop sufficient data to justify its 
continuance or expansion. 

The fourth purpose for evaluation recognizes the reality of the 
new industrial state — the corporate society. Such a state produces 
hierarchical bureaucracies (this phrase is, 1 suppose, redundant) in 
industry, government, labor unions and volunteer organizations as 
well as universities. Evaluative reports are, of course, necessary 
for the proper functioning of the enterprises which are the concern of 
these bureaucracies, particularly for the use of the technostructure, 
as Galbraith has called the decision-making groups in large industries. 

The reader is reminded, however, that bureaucracies lead a 
life of their own that is somehow magically related to the flow of paper 
in and out of little wooded or wire baskets and conferences in confer- 
ence rooms concerning the leapfrogging of this paper among the bas- 
kets. Paper, then, must be generated so that the system may lead its 
organic, inward life. Evaluation studies may be a large part of this 
pulsing circulatory system — the corpuscles, so to speak. It is 
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emphasized that the relation of this particular form of corpuscular 
paper with the real, operational world may be nil or almost nil. In 
many cases that is not its purpose of existence. 

It then follows that a careful distinction must be made between 
required evaluation which is necessary and has an effect on operations 
and decisions and that which only serves the life function of the bureau- 
cracy itself. The first needs improvement; the second needs to dis- 
appear. 

Finally, the evaluation process is undertaken to provide data 
on new developments in order that these data may be diffused through- 
out the educational community so that schools in distant places may 
understand and take advantage of the findings. This idea is a little 
tricky, as it could be held that the evaluation comes first and diffusion 
follows as a matter of course. In many cases this is, in fact, what 
happens. However, there are other cases in which the distinctions 
between evaluation process and diffusion instrument are not so clear. 
The generators of a good idea want to sell it. The evaluation can be 
the package. Obviously, such a package may not be the same as an 
evaluation package for researchers, decision-makers or bureaucrats. 

These broad purposes for evaluation do not coincide very well, 

I fear, with the meticulously drawn detail in the charts of the experts. 

It may be that their only value is in delineating the perceptions of the 
responsible administrator on the firing line. Thus, for example, if 
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a request for evaluation is seen as necessary for t ie functioning of 
bureaucratic life, it will be developed with that purpose in mind — a 
useful paper corpuscle designed for the bureaucratic arteries, not 
for real time operations. 

It may be more important, however, to examine the drive to- 
ward systematic evaluation in the Title III program from the point of 
view of these five broad categories of evaluation purposes. Evaluation 
of Title III for what? For diffusion, for checking and adjusting ongoing 
processes, for substantive knowledge, for justification — perhaps of 
the entire Title III program itself — or for improving the circulation 
of a bureaucracy? I believe that the mix of these purposes must be 
carefully measured before an intelligent judgment can be made con- 
cerning any agreed-upon evaluation procedure. 



The Problem of Institutionalization 
Leaving the question of purpose open for the moment, we can 
turn to what I believe is the heart of the matter, namely, the question 
of institutionalizing the entire Title III evaluation process. 

If this is so, then the arguments introduced in the introductory 
material could stand further examination. It was stated that systema- 
tic evaluation was considered desirable; that this was part of the 
general drive toward analysis, synthesis and systemization within the 
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educational culture; and that such systemization had its bad aspects 
as well as its good aspects. 

Although I have nowhere seen the concept verbalized in a pre- 
cise manner, it seems clear that we are being asked to provide guide- 
lines for the institutionalization of evaluation for Title III projects. 

The reasoning seems to go something like this: (1) present evalua- 

tion procedures are not good; they are spotty, at times sloppy and 
unscientific; many times they imitate the researcher's controlled 
experiment when they should be providing the decision-maker with 
feedback information to correct the system, and a credibility gap 
exists on the diffusion front; (2) "hard data" must be developed for 
public bodies at all levels concerned with the Title III program; (3) 
the rather bad evaluation procedures now in use are neither generating 
hard data nor helping in decision-making due to lack of knowledge, 
guidelines and skill among the operators of Title III projects; (4) by 
a thorough tightening up on the evaluation guidelines to be developed 
by experts, and by institutionalizing these guidelines with a system of 
information and controls, the evaluation procedure will be helped, 
public bodies will be made happy by the presence of hard data and bet- 
ter decisions will be made in directing ongoing projects. This is, in- 
deed, an enticing picture, and to raise questions about its fundamental 
premises seenns to be akin to questioning the institution of nnotherhood, 
Sigmund Freud to the contrary. 
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However, I would like to question the entire concept of institu- 
tionalization of the Title III evaluation process and insert into the re- 
cord a few arguments that might at least suggest institutionalization in 
a different form. It is granted at the outset that, with over 1, 200 Title 
III projects on which a considerable sum of money is being spent and 
with the great need to develop viable educational innovations which can 
be adopted by the educational community, improvement in evaluation 
procedures is a necessity. Further, operations on the scale of Title 
III require quality controls which are only made possible by large 
scale systematic evaluation procedures. 

Granting all this, questions may still be raised and arguments 
considered. First, while the analyses of the experts — Guba, Stuffle- 
beam. Stake, et. al. — of the evaluation process are impressive and 
potentially fruitful, is it possible that they have, in fact, over- analyzed 
the process and, in doing so, slipped into the same trap that the con- 
ventional educational research man does when he attempts to apply 
controlled research techniques to evaluation processes operating under 
field conditions? Are these analyses, rather, important additions to 
our substantive knowledge and should they, instead, be used to generate 
more study of the process so that field applications would eventually 
develop? Have, in fact, these analyses departed from operational 
reality, at least in the sense that the practitioner would not know what 
to do with them? And, if one or more of these models was frozen into 
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6nforc6d guidelines, would this not result only in bureaucratic paper? 

I am not sure as to the answers to these questions, but I feel that 
these possibilities deserve more consideration than they have been 

getting. 

I have no question, however, on another point. The proposed 
models simply do not embrace all Title III projects. There is a tend- 
ency to forget that a portion of the Title III effort is designed to pro- 
vide supplementary educational services to various geographic areas, 
and proposals have been submitted and projects funded for such ser- 
vice centers. Further, entire new educational program efforts ("A 
Six County Program in the Performing Arts") do not lend themselves 
too well at first to measures that are meaningful and always present 
difficult problems for evaluative information systems. 

The service centers present the real challenge, however. 

There has been, I believe, a tendency on the part of the USOE to play 
down, if not ignore, the meaning and importance of the service center 
concept to units of the educational community. This is due, no doubt, 
to the decision which placed the emphasis on the innovative aspects 
of Title III rather than on the supplementary service center idea. It 
is still a fact, as noted above, that some of these centers have been 
approved and funded, and if there is to be systematic evaluation for 

Title III, these projects must be included. 

In this connection, there are two problems. The first problem 
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is somewhat technical. The difficulty lies in the fact that the sophisti- 
cated evaluation models do not exactly fit the problem of evaluating a 
service center — for example, a media center supplying media services 
to a group of school districts. These models, for all their claim to 
generality, tend to concentrate on innovation in the instructional pro- 
cess — curriculum, methodology, the mediation of instruction. It 
seems pretty obvious that when you set up a service center of some 
kind, the distance between the regional center and the student is highly 
attenuated from the point of view of evaluation — both as to time and 
distance. To expect the evaluation process as abstracted in these 
models to cut through from a regional center to a student in the fifth 
grade of George Washington school in one of six school districts, de- 
fine the effect that the sudden acquisition of a film library had upon 
him in a year's time, and adjust the content or service of the library 
accordingly is also to expect that the films will be delivered to the 
school via flying saucers piloted by little green men. 

I should hasten to add that the principles inherent in the models 
can be, in many cases, applied to the evaluation of service center 
operation. The problem is that, if the institutionalization of the evalua- 
tion process continues to proceed and harden along the lines it is ap- 
parently proceeding, harrassed administrators will be asked to eval- 
uate a service center operation by standards that ought to be applied to 
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the evaluation of a new approach to phonics in the teaching of reading. 
This simply would not make sense. 

In addition, the media world is not without a certain sophistica- 
tion in the evaluation of service center operations; arid these proce- 
dures relate to the principles enunciated by Cuba, as one model 
maker, but not to the tactics that seem to be implied. To cite a homely 
example, if film keeps coming back into the library from a given dis- 
trict all chewed up, the center director then has a practical measure 
readily at hand which requires further investigation immediately. He 
must find the answers to such questions as: How are the films pro- 
jected? By students, teachers or both? If teachers, are they doing 
it properly or do they need some training? If the human factor is not 
the problem, what about maintenance of the equipment? If the provi- 
sions for maintenance and control are all right, what about the per- 
formance of the Acme Repair Company on the projector service con- 
tract? Etc. Etc. Once these questions are answered, then changes 
can be made. This procedure is in line with the principles of Cuba, 
Stufflebeam,j^.^., ,but not the suggested tactics that seem to flow from 
them. Problems such as this one will only become difficult if evalua- 
tion is institutionalized. Under such hardening of the categories, 
the evaluative universe is interpreted to be the instructional process 

and the responsible administrator is forced to proceed accordingly. 

* / 

A phrase picked up from Edgar Dale many years ago. 
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The second problem defies the models. An examination of pro- 
posals for Title III centers (mainly media service centers), both funded 
and unfunded, shows immediately that the funds are very badly needed 
to supply materials, equipment and services that are sadly lacking in 
the districts to be served. NDEA funds, articles about media, and 
fears about commercial domination to the contrary, the plain fact is 
that many, many schools in this country do not have enough of anything 
to do the job required of them. Under Title III they get some money 
for equipment, materials and services. It is like giving a drink of 
water to a man who has spent three days on the desert without it. How 
are you going to evaluate that? By the test of survival? What is sur- 
vival in the educational setting? What these few centers funded by 
Title III mean is that all of the schools involved are experiencing an 
increase in their technological base. I submit it is only after this 
base has been functioning to the point where it requires additional 
technology does evaluation become meaningful. In the beginning, any- 
thing is better than nothing. There are, of course, still evaluative 
questions, generally pointed toward improving operations and attaining 
efficiency. Other models, however, are needed for this. 

In a sense, the broad educational programs mentioned above 
fall into the same category as the service center. In some of the pro- 
posals I examined, for example, broad programs in the performing 
and plastic arts and the humanities were proposed for regions which 
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had absolutely nothing of this kind but the prints sold at Woolworth's, 
the local piano teacher and the county pioneer pageant at the fair each 
fall. In one rather large area, for example, there was no school ex- 
cept a religious high school where a student could get instruction in 
the playing of any stringed instrument. Again, the desert-water an- 
alogy holds. Some things are obvious. Music, art or the theater 
brought into a community make things better, period. It seems, in 
a way, ridiculous to measure or count such efforts; members of Con- 
gress should be happy with the invasion of the arts as a happening; 
experimenters or journalists might have to wait a few years for ex- 
perimentation or diffusion. It is granted that the best possible opera- 
tion is needed, but a narrow institutionalization of the evaluation pro- 
cess for Title III projects will not provide that better operation. 

If the evaluation theory we are apparently following does not 
exactly fit the service center and large program projects, in incon- 
gruence must show somewhere. It does in reference to objectives, 
an important point to notice when thinking about evaluation. Almost 
every reference to objectives (the achievement of which are to be mea' 
sured) refers to behavioral objectives or some variant thereof. In ad- 
dition, performance tests, criterion tests, etc. are easily picked up 
bywords in discourse on evaluation. It is as if the jargon of program- 
med instruction has suddenly become the lingua franca of all educa- 
tional evaluation — or, for that matter, of all education. 
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Now, some of my best friends are behavioral objectives, bat I 
would not want my media service center to marry one. Seriously, ob- 
jectives are one thing and behavioral objectives are another. Behavioral 
objectives are a microcosm, to be entered into when students are di- 
rectly related to content, processes, media or people in the class- 
room. To apply them to large programs embracing all of the arts 
throughout a wide region, a library service center for a county school 
system, or a data processing installation represents a beautiful con- 
fusion of form with substance— setting up the conditions of operation 
for an educational Parkinson's Law. Even smaller sectors of the 
educational enterprise directly related to instruction may not need 
objectives stated in behavioral terms. 

I wish to make it very clear that I am not attacking behavioral 
objectives as su-ch. They can be made to accomplish spectacular things 
with certain instructional processes and are legitimate targets for 
evaluation. On the other hand, sometimes we will need system objec- 
tives, which are not the same thing at all. By stretching a point, it 
might be said that an evaluator might want to measure (and change) 
the behavior of a system (such as a library, full-scale curriculum 
operation or something else), but I do not believe that this type of ob- 
jective was exactly what B.F. Skinner had in mind (or Ralph Tyler 



many years earlier)). 

There is, of course, nothing in the models with which we have 
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been dealing that requires behavioral objectives; and it is also a 
truism that evaluators can't evaluate for any purpose without objec- 
tives. The fact remains that all the discourse about evaluation is con- 
ducted a^ji there were no other types of objectives in the educational 

universe, even when the discussants unconsciously know better . Thus 

the incongruence between the theory (or theories) of evaluation under 
analysis and the real time world of operation can be shown to be a 
possibility. 

This exploratory discussion relating to some specific problems 
of institutionalizing Title lU evaluative processes can now be brought 
into focus at the philosophical level. To review, there is apparently 
great concern as to the quality of existing evaluation, there is a de- 
sire to produce "hard data" for persuasive purposes, there is a need 
for accurate information as to progress and to adjust for improvement, 
and there is the necessity of diffusing information on successful prac- 
tices — success being determined by competent evaluation. Further, 
the size of the Title m effort (over 1, 800 units) and its wide distribu- 
tion geographically with enormous differences in the resources and 
abilities of the educational units involved, all press for standardiza- 
tion (at an acceptable level of competence) of evaluation procedures. 

There is, however, a deeper drive involved in this effort or, 
at least, I believe it to be so. The industrial state is the corporate, 
bureaucratic state. The imperatives of technology, we are reminded 
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by many observers such as Galbraith, have replaced ideology in much 
of our culture. Technology requires large scale organization, orderly 
processes, group planning, and, where possible, it seems to me, a 
kind of neatness in the system that one might associate with a computer 

installation or a "clean room" in an electronics factory. 

In a fundamental sense, the entire Title III effort at educational 
innovation is a move toward bringing the educational enterprise into the 
modern industrial state. Title U1 is educational technology. This may 
sound strange to those educators who define educational technology as 
a term synonymous with language laboratories, computers or televi- 
sion. I would remind them of Galbraith's definition of technology, al- 
though many similar definitions might be cited. Galbraith said, "Tech- 
nology means the systematic application of scientific or other organized 
knowledge to practical tasks. ^ He goes on to point out that the main 
characteristic of technology is the breaking down of tasks into detailed 
sub-divisions so that organized knowledge may be put to work, and that 
this analytical procedure "is not confined to, nor has.it any special 
relevance to mechanical processes."—^ I submit that the very selec- 
tion of evaluation as a field to analyze in connection with the Title III 
program is evidence of this movement toward technology. 

-^John Kenneth Galbraith, The New Industrial St^^ Boston: 
Houghton Mifflin Company, 1967, p. 12. 

9 / 

-^ Ibid . . p. 13. 
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In any of the units of a society of high technology, such as the 
United States, very extensive planning is necessary. Since the units 
are very large, for the most part (Galbraith's 500 "mature corpora- 
tions, " large government, etc.), the planning affects and controls 
millions. A systemization of the Title III evaluation process is a form 
of planning . This concept leads us to the crux of the argument. Gal- 
braith has noted, for example, that ". . .planning involves, inevitably, 
the control of human behavior. The denial that we do any planning has 
helped to conceal the fact of such control even from those who are con- 
trolled. He was speaking of economic planning, but I believe this 

concept to be totally generalizable in our technological culture. 

Planning involves the creation and management of systems; 
systems require, or at least imply, bureaucratic control. Hence, un- 
less, as indicated in the earlier portion of this paper, additional 
means are invented to fit the peculiarities of the institution with which 
we are dealing — the American educational enterprise — the develop- 
ment of a systematic, technically competent evaluation process for 
Title III will result in bureaucratic control that I believe would mean 
the end of the dream that Title III would bring needed innovation to 
American education. 

Such a prospect is difficult enough, but further complexities 
must be examined. Galbraith has pointed out that, with high technology 
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“^Galbraith, op. cit . , p. 23. 
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and large organization, as in the mature industry, the planning and 
management processes are in the hands of fluid groups of experts, 
each bringing complex information into the group processes where 
decisions are made. The fluid groups he calls the technostructure. 

We come now to the rub. American education, ^ secto r o| pol- 
itical economy , is very primitive from a technologica j. poin ty 

and has practically no techno structure. 

If the concept of a lack of a technostructure in American educa 

tion is accepted, it is possible to explain many things.—^ Galbraith 
does this in another context when explaining why socialist countries 
have had "the most uniformly dismal experiment of countries seeking 
economic development."—^ Speaking of India and Ceylon, he goes on 

to say that, in these countries, 

... if the minister is to be questioned, he must have knowl- 
edge. He cannot plead that he is uninformed without ad- 
mitting to being a nonentity. . . Technical personnel are less 
experienced than in the older countries. Organization is 
less mature. These lead to error, and suggest to parli- 
mentarians and civil servants the need for careful review 
of decisions by higher and presumably more competent au- 
thority. Poverty. . . calls for further review. And rigid 
personnel and civil service rules, the established British 
answer to primitive administrative capacity, extend into 



—^Consider the inability of the old line- staff administrative 
patterns to handle aspects of the new educational technology (hardware 
and materials logistics, etc.); consider the problems of the ghetto 
from this point of view; etc. 

—^Galbraith, o£. cit. , p. 101. 
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the public firm and prevent the easy consitution and re- 
constitution of groups with information relevant to chang- 
ing problems. 

It seems to me that it is easy enough to transfer this Galbraith- 
ian concept to the American educational system. To begin with, there 
is no large scale organization in the technological sense and, as noted, 
no technostructure. If development is to occur, it becomes obvious 
to those responsible — in our case, the USOE and our study commit- 
tee — that review and control of decisions and operations relating to 
evaluation are absolutely necessary when dealing with such "primitive 
administrative capacity" — and it ^ primitive from this point of view. 

At this point, however, it is necessary to exorcize a ghost, I 
am suggesting that national bureaucratic control or even systemization 
of evaluation seems to be necessary under the circumstances, but I 
am further suggesting that this may be unwise (the reasons for this 
will be discussed below). It then mi follow that all I am interested 
in is a reduction in the size of the bureaucracy and the removal of the 
controls to the state level. Nothing could be further from the truth. 
Bureaucratic control from the state level will merely extend the "pri- 
mitive administrative capacity" from the school district upward. I 
venture to say that no state department in the United States has an 
adequate technostructure or is about to get one — and this includes 

13/ 

— ^Galbraith, op. cit . , pp. 101-102. 
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New York and California, both of which have been praised in many 
quarters. I believe that institutionalizing Title III evaluation processes 
under state departments of education will concentrate many undesir- 
able elements of such a system. My arguments against institutiona- 
lization in the form that seems to be implied by events must be seen 
in this light. Once the arguments are considered, it may be possible 
to suggest a better solution. 

To return to the main theme, given the assumption of planning 
and systematic (translate bureaucratic) control of Title III evaluation 
procedures, certain undesirable effects seem to inevitably flow from 
many (but not all) such developments. High technical solutions to 
some problems require exactly such arrangements — getting to the 
moon, stamping out an epidemic, etc. In such situations large scale 
technology and its peculiar requirements seem to fit fairly well and 
the people involved are relatively comfortable. However, all large 
scale applications of technology (systematic organization) do net fit 
particularly where they impinge in certain ways on human beings (re- 
call Galbraith's sentence on control). 

This lack of human fit of many of the technological develop- 
ments in the United States — depersonalization of university life, smog 
and the automobile, social decisions made by the corporate structure 
over which those being decided about have no control has given rise. 
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in the last decade, to a heated dialogue which has erupted on numerous 
occasions into violence. 

It is a mistake, however — a serious mistake to view the dia- 
logue only from the point of view of the violence or certain individual 
issues such as Viet Nam, civil rights or rent strikes. For anyone who 
cares to take the time to inquire, a much deeper dialogue, a much 
deeper emerging philosophical statement are there to hear. "To hear" 
is used advisedly, for I believe unless we listen to what some of these 
bright young people are saying, to what the New Left is trying to ex- 
pound, to what some artists are expressing, we, as educators, may 
fail this country and all of the young people in it. 

What is this dialogue? It is a dialogue between high-order 
technological organization, the industrial state, impersonal controls 
over people and spokesmen, no matter to the degree that they are right 
or wrong, for men as human beings — for man in microcosm. An edu- 
cator, it seems to me, does not necessarily have to adopt totally the 
view of one side or the other. Some educators, at least, ought to see 
the thousand dilemmas present in this confrontation and seek solutions 
which are, first, educative, and secondly, human without reducing our 
culture back to some primitive stage where we live in the hills in 
shacks. I believe we should seek in general what William Javanovich 
saw in the future when he predicted "the emergence of a new kind of 
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intellectualism which will reconcile content with style, social purpose 

nil/ 

with personal sensibility. 

Assuming that I am right in understanding that the effort to 
"improve" the evaluation processes associated with Title III will move 
in the direction of national systemization and control (or, worse, state 
systemization and control), the criticisms of these spokesmen for the 
defense of man as man have relevance. They should be seriously 
thought about, for, within the intimate environment in which each man 
iives, they attack systemization and control with a vengeance. 

Let us begin with one of the best known spokesmen for this point 
of view, Paul Goodman. Recently he was asked to address the National 
Security Industrial Association and took the opportunity to berate this 

industrial-military technology group. At one point he said: 

Your thinking is never to simpiify and retrench, 
but always to devise new equipment to alieviate the mess 
that you have heiped to make with your previous equipment. 

And, then he went on: 

Your systems analyses of social problems always 
tend toward standardization, centralization, and bureau- 
cratic control, although these ^ not necessa.ry; 1kg- 
method , (italics mine) 

Finally, he stated a principle or theme that reappears time and time 
again in this literature: 



—'^William Javanovich, "My Illusions and Yours, " Harpers , 
Volume 235, Number 1409, (October, 1967), p. 59. 
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In a society that is cluttered, overcentralized, 
and overadnninister ed, we should aim at simplification, 
decentralization, and decontrol. J^/ 

A great deal of this new literature is being created by young 
people. A whole issue of the American Scholar was recently devoted 
to writing by people under thirty. In it, Michael Rossman made an ef- 
fort to explain the deep philosophical base of the so-called National Stu- 
dent Movement. In doing this, he expressed much about their concern 
with man as individual man and even explained (and this is a little hard 
for an older person to understand) "participatory democracy" both as 
philosophy and as tactic. Three concepts appear in much of this litera- 
ture, and they appear in Rossman. They are: Engagement, Encounter, 

and Involvement. The concern is with humans relating to humans — 



with true encounter . Rossman puts it this way: 

. . .the present Old Left among us. . .aims. at the mass; 
at the racial, economic or occupational population. But 
the unit in terms of which the Movement conceives change 
tends to be the small group. 

... The way to influence large groups is by local example, 
rather than global persuasion. 

. . . direct personal involvement is the Movement’s human 
backbone. 

. . . In saying that people must be involved in the decisions 
that shape their lives, the emphasis is on involved . 



— ^Paul Goodman, "A Gauserie at the Military-Industrial, " 
New York Review of Books. Volume 9, Number 9, (November 23 , 
1967), pp. 16-17. 
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. . . political dialogue must be cast in a different vocabu- 
lary than that possible with the comfortable separation 
of the Changer and the Changed.-^^— ^ 

Harper’s recently published a symposium consisting of a series 
of dialogues between well-known older commentators on the national 
scene and a panel of young people similar to those appearing in the 
American Scholar . In this case, it was, at times, hard to distinguish 
between the older and younger viewpoints. Again, the theme reap- 
peared time and again. The issue was imper sonalism, bureaucratic 
control versus general encounter and the human condition. It is tempt- 
ing to go on quoting a great deal because the material seems so rele- 
vant, but I shall try to restrain myself. Paul Potter, one of the 

"older" members of the panel, said: 

. there is a growing belief that the only force really 
shaping the future is the force of unleashed technology 
controlled by giant, impersonal bureaucracies. 

. . . economic planners cluck truculently about the "great 
leveling force of technological development" that will in 
time assimilate all revolutions and all cultural diversi- 
ties into one grand machine civilization..!.®./ 



—^Michael Rossman, "The Movement and Educational Re- 
form, " Th^Am^rm^Schol^ Volume 36, Number 4, (Autumn, 

1967), pp. 595-596. 

— ^It should be noted that, for the professional educator, Ross- 
man has some provocative things to say about teaching and learning in 
higher institutions and proposes some interesting reforms. 

— ^Paul Potter, "The Future is Not Inevitable, " Harp^rjs, 
Vplume 235, Number 1409, (October, 1967), p. 48. 
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In commenting on Potter's article, young Robert Gross said: 

... we have to end the domination of this society by the 
large, rigid bureaucracies which pay little attention to 
the needs of the people they are intended to serve 

And Alfred Kazin commented: 

. . .the more immediate and abundant our technical power, 
the more we lose the naive, spontaneous imagination 

Other parts of the text refer to "students who are demanding 

flexibility and personal relevance, " "non-rational ways of getting at 

knowledge," "taking strength from the free private life." There is 

also, however, another thread which suggests that something better 

might be made of this "technocratic totalitarianism, " as Potter called 

it, and he went on to say: 

The technology and the bureaucracy can be mastered 

and put to work to create for everyone what we've begun 

^ 71 / 

to have a taste of. . 

A Suggested Ac commodatioii 

The argument has come full circle, and the potentials of ac- 
commodation are there if they can be identified. It seems to me im- 
portant to suggest a new approach to the problem of institutionalizing 

—^Robert A. Gross, "To Mr. Potter." Harper'_s, Volume 
235, Number 1409, (October, 1967), p. 50. 

—^Alfred Kazin, "Art on Trial." Harper's , Volume 235, Num 
ber 1409, (October, 1967), p. 51. 

—^Potter, op. cit . , p. 50. 
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evaluation for Title III projects which would accommodate need for and 
technique of consistent, high-quality evaluation procedures with human, 
local needs and differences in projects and concepts. Personally, I 
feel that many of the critics of technical bureaucratic control cited 
above offer little replacement for this control with a sort of leaderless 

t 

•'participatory democracy" which, in a technical sense, will not even 
achieve their own objectives. And yet, much of what they have to say 
is important. 

We have been concerned with models of high-quality evaluation 
procedures; with purposes of evaluation; with implied arrangements 
to insert controls in the system so that legitimate purposes may in- 
variably be supported with technical competence. All of the elements 
of a bureaucratic system are there — whether in the eventual rough and 
tumble of administrative or legislative politics this control is placed 
at the federal or state level. Of course, such control could apparently 
be non- enforced by guidelines or some other system which in fact 
would quickly encrust into a strait-jacket. On the other hand, controls 
are needed so that competent, useful evaluation may take place. This, 
to repeat, is the problem of accommodation. 

And I hold that it is a solvable problem and that the possible 
solution, as Boyd Bode used to be fond of saying, "lies at hand. 

Many of the elements are present in Project EPIC of Tucson, 
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Arizona.”~ Project EPIC is, essentially, a sort of local evaluation 
service center funded by Title III funds and assisting local school agen- 
cies within the area it services. 

Given this idea as a start, it is possible to make a series of 
recommendations that can, I believe, achieve the sought after accom- 
modation between the need for evaluation and the human variation which 
inevitably occurs at the end of the line. Such an accommodation will 
not be as neat as a clean room in an electronics factory; on the other 
hand, it will not be so messy as to be useless; in fact, it might have 
enough variation in it to release creative energy — which was the 
general idea of Title III in the first place. 

Recommendations 

1 . Title III funds be used to set up _a series of regional 
evaluation centers throughout the United States designed to provide- 
training and assistance to local educational agencies . 

2 . The function of these centers be ^ provide advice , 
training and services and , particularly , to diffuse the general idea of 
the importance , usefulness and nature of _a high - quality evaluation sys - 
tem . 

3 ^ understood that the evaluation centers are only per - 

suasive and helpful in nature and that , if an educational agency chooses 




Hammond, op. cit . 
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not,^ _res£Ond, it be allowed to without penalty — actual or implied . 

4. These centers also engage in a certain amount of applied 
and field research with the purpose of developing viable and variable 
evaluation procedures which can embrace all types of evaluation needs 
and purposes . 

5 • A back - up national board be set up to assist the centers 
and the USOE and Congress . This board would have the following func- 
tions ; 

a. Locate and rotate manpower between the centers. 
Much of this manpower could be one-year leave-of-absense 
type; other slots could be filled with qualified graduate stu- 
dents on an intern basis. 

b. Act as the assembling agency for results which 
ought to be diffused and as the communication agency between 
the centers. As such it should act as both the stimulus and 
the conscience for the centers. 

c. Engage in broad scope research and development 
studies in the field of evaluation. 

d. Provide an information source for all government 
agencies, local, state, federal. 

e. Relate to and diffuse information to the educational 
community about other national, private evaluative efforts, 
such as the National Assessment Program, etc. 
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Under no circumstances should this board be thought of as a 
control mechanism in the bureaucratic sense. 

If these recommendations are analyzed for the purpose for 

j 

which they were made — to create a system which would achieve the 
objectives of necessary high-quality evaluation procedures for local, 
human purposes without inserting another bureaucracy into the sys- 
tem, details of operation and administration should become reason- 
ably clear. The human being at the end of the line — administrator, 
teacher or media specialist — can have his opportunity for involvement 
and encounter. And it is highly likely that we can raise the quality of 



evaluation immensely. 



PACE EVALUATION 



^ ! 

by Robert E. Stake” 



Public Law 89-10 Title III has had a sweeping effect upon what 
U. S. schools are doing for children. New opportunities for learning 

t run into the thousands. The number of children who have benefitted 

i 

those who are wiser, or happier, or more enthused about learning 
runs into the many thousands. According to the wording of the legis- 
lation, Title III establishes a program 

"to stimulate and assist in the provision of vitally needed 
educational services not available in sufficient quantity, 
and to stimulate and assist in the development and estab- 
lishment of exemplary elementary and secondary school 
educational programs to serve as models for regular 
school programs. " 

This has happened. Title III has enhanced and increased educational 

I opportunity for many children. 

Title III of the Elementary and Secondary Education Act of 1965 
enables local schools to use federal moneys to increase instructional 



jjC / 

— 'Associate Director, Center for Instructional Research and 
Curriculum Evaluation, University of Illinois. 
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services. Either through regional (e.g., multi-county) supplementary 
centers or through projects within the local school system, the school j 

could use materials or techniques not otherwise available to it. Fund- 
ing was expected to be transferred from federal to local sources, 
usually after three years. 

As of February, 1968, the U. S. Office of Educeition had ap- 
proved 2, 265 projects in the 50 states and territories of the United 
States. About 45 percent of these were planning projects, most of 
which were (or will be) followed by separate operational projects. 

The funding as of that date had exceeded $200 million. 



Evaluation 

There are three ways to look at the responsibility for evaluating 
a large social endeavor like Title III: (1) describe what it is doing 

and how well people like what it is doing, (2) contrast what it is do- 
ing with what it could be doing under another plan, and (3) study its 
components to learn how better programs might be assembled in the 
future. Little of the data gathered for one purpose would be likely to 
be of much value for the other two. 

Most of the attention which has been given to the evaluation of 
Title III by the U. S. Office of Education, by project personnel, and 
by interested bystanders is pertinent to the first part of the first point 
above. The emphasis on evaluation in project proposals and contracts 





















mmmmmmm 



- 197 - 

concerns the effects that project personnel can report. The more in- 
teresting and useful aspects of evaluation having to do with value- judg- 
ments and decision-making are given little attention. Evaluating Title 
III as a whole-greater- than-the sum-of-its-projects is at best a fleeting 
wish. 

As a member of the Committee of Special Consultants (chaired 
by Richard I. Miller of the University of Kentucky), I have been asked 
to discuss what is being done, and what could be done, to evaluate 
Title III. I have chosen to include a number of value- judgments of my 
own regarding the quality of Title III activities. The next few para- 
graphs will display those opinions; the remainder of the paper will 
consider the evaluation responsibility in general. 

U. S. Education — as an enterprise — has not been reshaped by 
this federal legislation. More good things have been done — many new 
things have been done — but education as a social enterprise is not 
discernibly different in a Title III school from education in a non- Title 
III school. Some citizens cry out for revolutionary change. Title III 
purports to be the instrument of innovation, but it seeks to do nothing 
about changing the purposes or changing the management of the schools. 
Congress had no such intent. 

Many citizens think that critical innovations in education are 
obstructed by the existing professional establishment. In Six Urban 
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School Districts : A Comparative Study ^of Institutional Response , 

Marilyn Gittell and Edward Hollander said: 

’’The insulation of public education is twofold: bureau- 
cratic centralization. . . reinforced by an ideological ra- 
tionale of professionalism. . . The result is a static, in- 
ternalized, isolated system which has been unable to 
respond to vastly changing needs and demands of large 
city population. ” 

Such observers are not favorably impressed by gains made by Title 
III. They are more sympathetic to gains made in programs of Title I 
and the Office of Economic Opportunity, but they are not satisfied by 
them either. Many citizens yearn for schools to lead the way in break- 
ing down the monopoly of privilege held by the upper and middle classes 
of our society. Title III has extended some important privileges to the 
educationally disadvantaged, but it is no instrument for the renovation 
of social values. It was not intended to be. Especially in its amend- 
ment of 1967, the law supports the same viewpoints on allocation of 
privilege that have prevailed for decades: reward accrues to those who 

excel. The alternative holds that all should share in the rewards of an 
effective economics system, that all should share in the benefits of a 
secure community, that all should share in opportunities for education. 
This alternative is preferred neither within the professional ranks nor 
outside education today — and nothing in Title . Ill pronriotes a renovation 
of these values. 

A majority of Americans see American education — its 
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constitution and its values — as essentially sound. They see it staffed 
by reasonably trained, well-meaning men and women. They see it as 
very expensive, hard pressed to make do on the funds it has, barely 



able to venture out in any way. Title III has provided "venture capi- 
tal. " These people can take a measure or pride in Title III because 
it has enabled some of the best ideas of our school people to be blue- 
printed and engineered. 

The preponderance of educators who have studied the effects of 
this portion of the Elementary and Secondary Act are convinced that 



this federal money was a "shot in the arm" — an opportunity to inno- 
vate, to collaborate, to obtain help from specialists, to "stretch out" 
in ways that the schools have never previously been able to afford. 
Although most of these observers are aware of isolated instances of 
bumbling and misguided effort within the program, the consensus is 
that Title III moneys have been spent on a sensible purchase. Whether 
or not we are getting a bargain, and whether or not we should purchase 
more of exactly the same are matters of some disagreement, but there 
are few educators who would recommend large changes in Title III. 



Nothing persuades me that Title III has brought a new brilliance 
or tenacity or compassion to the classroom. Brilliant, persisting, 
compassionate teachers are working in Title III projects. Wise, 
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creative, and dedicated people have taken jobs in education because 
Title III moneys are available, but so have persons of little talent. 

The extent to which education draws from the upper levels of the na- 
tional talent- pool has not been altered by Title III. The same can be 
generalized to include other supplementary programs. We do not 
have evidence that with this aid the education sector of the national 
enterprise operates at a higher level of effectiveness than it does 

without. 

Nothing in the studies of Title III activities suggests that U. S. 
education is spending these moneys more wisely or less wisely than 
the local school is spending the other moneys available to it. The 
rontinuation of support for Title HI — ^ this time i^^st re , s t mi t^ 
fact that it is er abling the schools to provide learning opportunities^ 
to innovate , as they otherwise would not, in ways tha^ ajlg. generaU _ y 
judged valuable by educationist and layman alike . Whether or not 
greater benefits would have obtained had the same moneys been used 

otherwise is something no one knows. 

It is reassuring to know that our national leaders are upset by 
the lack of better evidence. Peter H, Dominick, U, S. Senator from 
Colorado, has said: 

"As a legislator, I am vitally concerned whether /Title III 
projects? are in fact accomplishing the objectives envi- 
sioned by Congress and whether the money appropriated 
is being properly spent, . , I am specifically concerned 
about whether the programs conducted will have a lasting 
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effect on the school — or if, when the money for a project 
is exhausted and the initial program is terminated, the 
tent will be folded with little or no imprint left on the edu- 
cational processes of the school system. " 



And Patsy T. Mink, U. S. Representative from Hawaii, has advised us: 



. .1 hope to find. . . some of the evaluative information 
that is lacking now to enable us to judge the relative ef- 
fectiveness of Title III programs around the country. I 
recognize, of course, the inherent difficulty in applying 
any reliable measurement to those projects aimed at cul- 
tural enrichment, whose results are intangible and so 
diffuse as to defy evaluation. " 



Neither they nor the taxpayer nor the local PACE project dir- 



ector will find that information, that evidence, in 1968. The studies 



that might have given even shreds of evidence have not been under- 



taken. It is not reassuring to realize that we Americans have no bet- 



ter evidence of the impact and tactical wisdom of the Peace Corps, the 



U. S. Marine Corps, the U. S. Postal Service, the Ford Foundation or 



the Roman Catholic Church. Our judgments of them — and of Title 



III — will continue to reflect our values, our faith, and the apparent 



logic of their reasonings. Still, our judgment should be swayed by in- 



formation from the field, but systematic collection of that informa- 



tion is just barely under way. 



A most serious obstacle is the incompatibility of information 



from different projects. Partly because they have different purposes. 



but partly also, because they have different ways of expressing 
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common purposes and common results, the impact of Title III is dif- 
ficult to summarize. 

"School statistics as at present compiled and compared are 
unreliable and of little value, and they will continue to be 
so until agreement can be reached not only as to terms used 
and the definite meaning of these terms, but also, to some 
extent, as to the method of recording and arranging the 
original data upon which school statistics are based. " 

It is dismaying, on the one hand, to realize that these words, 
so relevant today, were written in 1912 by the U. S. Bureau of Edu- 
cation’s Committee on Uniform Records and Reports. But it is en- 
couraging, on the other hand, to realize that that Committee was 
concerned about the lack of uniformity in fiscal accounting in schools — 
a disorder that has been greatly alleviated in the intervening 56 years. 
Budgets, inventories, and personnel accounting are now sufficiently 
orderly that the administrator can understand his fiscal position; and 
the technical language is sufficiently defined and in vogue so that he 
may talk meaningfully with colleagues unfamiliar with his operation. 

Not so the instructional supervisor. He has only his personal experi- 
ence upon which to register the accomplishments of his wards and little 
more than the language of the layman with which to summarize their 
deeds. 

The same people who brought order out of chaos in the fiscal 
realm are attentive to the needs for achievement accounting. One 
agency, the Data Compatibility Group of the U. S. Office of Education, 
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is in the final stages of publishing a dictionary of characteristics of 
the learning situation. The authors, John Putnam and W. Dale 
Chismore, say: 

"The universal use of the terminology in /pu£7 handbook 
can improve the quality of education by facilitating mean- 
ingful evaluation, realistic planning, and efficient opera- 
tion of educational systems throughout the United States. . . 
Reports based on these items of information will not only 
merit the full confidence of their users but will be more 
easily interpreted and more useful. " 

Evolution of Title III Priorities 

But another obstacle to evaluation (of all socio- educational pro- 
grams) is the evolving character of goals and priorities. As social 
conditions change, as attitudes toward those conditions change^ and as 
ideas succeed and fail, we seek new approaches, we reallocate our ef- 
forts. It is to the credit of any institution that it is sensitive to chang- 
ing demands on it. PACE administrators have reformed and refined 
their statements of aim and priority. The evolution of Title III priori- 
ties (in an oversimplified representation) is shown in Figure No. ,1. Itis 
apparent that the priority statements have reflected the nation* s atten- 
tion. To many observers. Title III has been "keeping up." Its evolu-. 
tion of priorities is something to which it may point with pride. 

The evaluator tends to view such changes with alarm. He is 
sensitive to the difficulty of evaluating a program whose goals and 
priorities ebb and flow. With new goals, different performances need 
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to be measured, different methods of teaching need to be observed, and 
different effects need to be assessed. Some evaluative studies, once 
thought to be vital, become incidental even before their completion, and 



the studies now considered vital never got launched. 

Is there any hope for evaluation? Clearly, there is. The kind 
of evaluation that will offer some durably useful information will antici 
pate these changing conditions. Major studies must be broad, com- 
prehensive—— attending to changing cultural conditions and to changing 
aspirations of the people. 

Unfortunately, there is little respect among the USOE planners 
for broad, comprehensive, longitudinal surveys. The skills of the 
anthropologist, the social- survey specialist, the ecologist or the edu- 
cational historiographer are in little demand. They should be in high 
demand. There is at least some support in Congress for this kind of 
evaluation. U. S. Representative Marvin L. Esh from Michigan has 



said: 



"... I am not so certain that a mere recounting of the scope 
and methodology of. . . projects would contribute much to 
the information we most need in order to make legislative 
decisions. We could best use the kind of program evalua- 
tion information that could be supplied by an agency such 
as the Institute of Social Research of the University of 
Michigan. " 

Whether it draws talent from Michigan or elsewhere, the USOE could 

profitably broaden the spectrum of its evaluation in this manner. 
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Short - ys . Long - Range Investments 
The advocates of Title III are sometimes promising too much 
from it. They had said that today's pupils will get a better education 
and that the planners of tomorrow's schools will learn how to provide 
a better education. A double payoff is too much to expect. 

PACE is a developmental program. It supports the development 
of new teaching techniques. It is an investment in improving learning 
opportunities for the present generation of school childreyn. I am not 
questioning the essentiality of this investment when I call it a short- 
range investment. In the sense of educational planning and classroom 
instruction, Title III is a short-range investment. Its payoff is for the 
present generation, only very indirectly for subsequent generations. 

We educators hope to learn something from these Title III pro- 
grams. We are evaluating them partly because we want to understand 
what makes them succeed or fail. As prototypes, these innovative pro- 
grams are valuable — they give us something concrete to talk about. 

But they add little to our insight for designing programs. Insight is 
more than inspiration; it is a comprehension of the workings. If ver- 
balized, it is a set of generalizations about causes and effects. We 
should not expect that evaluations of Title III field projects will supply 
us with generalizations about optimal ways of administering supplemen- 
tary centers, or about providing novel opportunities for learning. In 
each of the field projects, hypotheses should be generated, hunches 
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tried out, and experience increased. These do lead to better efforts 
next time, especially under similar circumstances. But, valid and 
durable generalizations come from research studies— —studies aimed 
at the link between cause and effect — and seldom from developmental 
studies. Even the best evaluation of an innovation can leave the reader 
guessing as to what were the critical ingredients for success or failure. 
With extensive experience, or better yet, with controlled observation, 
the critical ingredients just may be discernible. 

Trying- out a good idea and researching a good idea are quite 
separate endeavors. No authorization was given in Title III for educa- 
tional research. If the nation wants to invest in schooling for the next 
generation of children, it should devote a substantially greater portion 
of Office of Education moneys to Title IV (Research) activities, and it 
should insist on a substantially greater allotment of Title IV funds for 
instructional research rather than development. 

The developmental activities now supported by Titles III and IV 
are designed for alleviating immediate needs, not for extending our 
comprehension of the processes that facilitate intellectual and social 
maturity. No evaluation of Title III— no matter how thorough, no 
matter how "hard-nosed" — will reveal what was not intended to be 
there in the first place — a systematic inquiry into the instructional 
process. 
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The U. S. Office of Education 

With matters specific to Title III, the Office of Education has 
been the middleman between Congress and the educator a coordinator 
of efforts, an auditor of accounts. Its major efforts with Title 111 have 
been to acquaint the profession with the intent of Congress, and to pro- 
vide guidelines for the expeditious handling of proposals. The Title 
III staff has appeared to accomplish these functions with care, some 
times with punctuality, and even occa3ionally with grace. As a group, 
the staff seems distinguished neither by wisdom nor ineptitude, but it 
does seem clearly distinguished by dedication to its task. These men 
are not the rattlebrained spendthrifts imagined by so many an anti- 
federalist; these are not the malevolent usurpers of local authority. 
These are schoolmen from Nashville and Willimantic, once wooed with 
a gossamer wage and the chance to do something for the good of the 
people, now beset with such myriad obligations and deadlines that 




nothing ever gets done quite to anyone's complete satisfaction. 

That does not leave much time for evaluation'. There are a few 
(perhaps three full-time equivalents) on the staff, especially assigned 
to evaluation; but evaluation there has meant, more than anything else, 
solicitation and processing of evaluation statements from the local pro- 
jects. They are caught by pressure from the National Center for Edu- 
cational Statistics to find out what changes in children a^ actually hap 
pening, by pressure from local educators to keep out of the way, and 
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especially by pressure to design forms and guidelines for fiscal years 
to come. They do not get around to doing any evaluation. U. S. Com- 
missioner of Education Harold Howe has had this to say about evalua- 
tors: 

•'Finally, I would season my stew with a dash of disquiet 
between the innovators and the evaluators. I think there 
should always be a slight uneasiness between these two 
camps, a feeling of discomfort like that between skepti- 
cism and faith. I‘m not saying that innovators and evalua- 
tors should not like and respect each other, but I am say- 
ing that they should stay on guard as they associate. If 
the innovators successfully capture the evaluators, then 
what the evaluators have to say won't amount to much. 

If they are captured, what they will be doing whether 
they realize it or not— is trying to find devious ways to 
prove that the innovators are right. One of the great 
shortcomings of modern-day educational innovation, to 
my mind, is that, by and large, the innovators have cap- 
tured the evaluators. " 

Not so with Title III in-house evaluators. They never were free to be 
captured to Start with. 



Antecedents at the F ederal Level 
For most of its existence, the main contributions to. evaluation 
of the USOE were its environmental surveys. It kept a tally of the na- 
tion's students, teachers, districts — the resources and settings for 
education. During the 1930' s, the nation developed a proclivity for 
conceptualizing education in terms of the individual child, and measure 
ment and evaluation were oriented to describing the standing of this 
individual within his group. Curriculum developers were pledged to 
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arranging educational experiences to meet his special configuration 
of needs. The teacher, as guidance specialist, and the academic 
counselor were the primary evaluators of the day. Such artifacts as 
course grades and standardized test scores were classified formally 
as evaluation, but contributed little to that activity. The federal 
government was not expected then to share in the movement; USOE 
took a minimal evaluation role. 

Later, in the 1950' s, when research funds were available from 
USOE, the range of environmental surveys stretched out to include 
sociological studies such as "Equal Opportunity?' and trait analyses 
such as "Project Talent. " Here still, the emphasis was on descrip- 
tion: descriptions of students and of the communities in which they 

lived. 

Educational researchers and technologists have been critical of 
environmental surveys that paid little attention to educational objec- 
tives and student behaviors. As such specialists accepted offices in, 
or consulted with, USOE; as the National Center for Educational Sta- 
tistics became more influential; as the nation became frustrated by 
its inability to see the "payoff" of its new investment in education; 
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guidelines for evaluation have increasingly emphasized behavioral ob- 




jectives and outcomes. Often the official — as well as unofficial — ad- 
vice to the project director has been, "specify your objectives, mea- 
sure the changes in behavior, and compare the program group to a 
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control group. ” These three steps have not been mandated by feder- 
ally supported projects, even when evaluation has been mandatory, not 
because the USOE program supervisors did not prefer it, but because 
they were not granted that much prerogative. Nevertheless, they 
echoed the call for "hard data" — objective measurements of behavioral 
change unequivocally related to treatment. USOE has sounded that call 
with frequency, and at times with fervor. 



The Quest for Hard - Data 

Among the research-oriented guidelines to evaluation have been 
Walbesser's (1965), Metfessel and Michael's (1967), and the Title I 
evaluator's manual, written by Neidt and French (1966). These aids 
focus on the one question, as Walbesser puts it, "What do we want the 
learner to be able to do after instruction that he was unable to do be- 
fore instruction? " They take the position that the value of a program 
does not exist independently of its impact on the individual learner. 

The focus of evaluation, then, must be on changes in pupil behavior. 
The honesty and good sense of this orientation are obvious. Neverthe- 
less, the advice is seldom heeded. The majority of evaluations of 
educational programs and institutions today do not give even secondary 
attention to student accomplishment. Evaluative Criteria (I960), the 
major checklist for secondary school self-study and accreditation, for 
all its thoroughness, does not even imply that it would be useful to 
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observe student productivity or to gather achievement test data. The 
First Annual Report (the 1965-66 evaluation of) Title 1 does not reveal 
one bit of objective information about what the disadvantaged learners 
actually learned. 

But the obvious need for evidence of behavior change is ig- 
nored, with some considerable justification. Educators have not 
found instruments to measure the changes that they believe do occur. 
Test scores are not easy to interpret. In terms of content or in terms 
of specific skills, scores on different tests are difficult to equate and 
to compare. Scores on most tests are so global that educators will 
seldom accept no- change as evidence of no learning. Furthermore, 
they will seldom accept measured change as evidence of worthwhile 
learning. 

A second reason why many educators resist the hard-data ap- 
proach is that they sincerely doubt that their purposes can be expressed 
adequately in terms of behavioral objectives. Atkin (1968) has objected 
to the preplanned specification of objectives, saying: 

"... the fundamental problem, as 1 see it, lies in the easy 
assumption that we either know or can readily identify the 
educational objectives for which we strive, and thereafter 
the educational outcomes that result from our programs. 

. . .we presently are making progress, toward thousands of 
goals in any existing educational program, progress of 
which we are perhaps dimly aware, can articulate only 
with great difficulty. . . ** 

Although some of the objection to behavioral preplanning might be 
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attributed to a desire not to be held accountable, the claims of such 
educators are not irrational. They are worried about the undesirable 
side effects of the hard-data approach. The behaviorists, like phar- 
maceutical houses, have a responsibility to show that the quality of 
scholastic achievement in our schools is not injured by a "behavioral- 
modification" commitment. 

From the evaluator's point of view, behavioral evaluation 
schemes are important components in any grand plan for evaluating in- 
structional programs. Taken alone, these schemes may be too inat- 
tentive to many of the complex but important phenomean of instruction. 
In attending primarily to the outcomes of education, these models tend 
to leave the audience with little knowledge of the instructional activity 
which brought about the intended and accomplished change. If, in a 
school, the "same growth change" is next year's objective, it is 
reasonable to prescribe the same setting and transactions. Most be- 
havioral-change models do not guide the educator toward making, 
setting and transaction a matter of record. Taken as they are, they 
are not easily coupled into an educational decision-making strategy. 

The Planning - Programming - Budgeting System 
Because there have been extensive claims of success for mili- 
tary cost- accounting, and because people everywhere are concerned 
about the soundness of federal expenditures, several forms of 
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economic summative evaluation have become prominent in discussions 
of planning and reporting federal programs. In July, 1967, Charles 
Schultze, Director of the Bureau of the Budget, provided offices 
throughout the federal government with guidelines for a Planning- Pro- 
gramming-Budgeting System (PPBS). Developed by a Department of 
Defense official, Charles J. Hitch, PPBS is a management information 
system which identifies alternate tactics for a given objective in terms 
of function and cost. Its program budgeting parcels out each expendi- 
ture, not to what was contractually purchased but to the outcomes at- 
tributable to it. The PPBS system analysis optimizes the ratio of 



benefits to costs. According to Hitch, the main purpose of PPBS is 

to make institutional decision-making both "feasible and realistic." 

"The function of systems analysis is to get dollars into 
the calculations at an earlier stage, into the planning 
process, into the evaluation of alternative ways of 
achieving /The/ objective." 

These procedures are expected to provide a statement of the costs of 
various benefits accrued. The purpose is a worthy one, but even the 
promise of using such a model— in the absence of an acceptable com- 
mon denominator of resource utilization (cost), and in the presence of 
multiple and sometimes contradictory goals (benefits) has not yet 



been demonstrated to the satisfaction of even a minor sector of the 
nation's educators. Their reticence would not be criticized by Hitch. 

He said: 
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"There are risks and dangers as well as opportunities In the 
application of new management techniques — including the 
risk of discrediting the techniques, if one tries to move too 
far too fast. Although it did not appear easy at the time, 
there is no doubt in my mind that the Department of De- 
fense, or much of it, is easier to program and to analyze 
quantitatively than many areas of civilian government. 

For example, it is certainly easier than the foreign af- 
fairs area. Quite apart from these difficulties, the sub- 
stantive problems in other areas are different and new. 

In Defense, we had several hundred analysts at the RAND 
Corporation and elsewhere developing progrsin's and sys- 
tems analysis techniques for a decade beforr the depart- 
ment attempted any large-scale general application." 

It will be especially difficult to "program" education. The 
benefits of education are not only many, but they run in greatly dif- 
fering patterns for different people. Effects considered desirable by 
some are undesirable to others. Are such effects costs, or are they 
benefits? And difficult though the criterion or output problem is, 
some observers, such as Malcolm Provus of the Pittsburgh schools, 
see the input problem as even greater. Getting dollar measures of 
what goes into education, especially the intangibles, and descriptions 
and classifications of the inputs is a formidable obstacle to the Plan- 
ning- Programming-Budgeting System. Obviously (and PPBS specia- 
lists are aware of it), no single, simple, cost-benefit ratio will tell 
much about a training activity or a federal program. 

Still, in the evaluator's search for new language, he would 
make a great mistake not to include economic indicators. How much 
a cost-benefit ratio can tell his audience is not known until his 
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audience has had a chance to validate it against other observations. 
(Unlikely as it may seem, the length of a column of liquid in a glass 
tube may be a good indicator of personal comfort, at least after the 
person has had a good chance to experience the covariation. ) 

The CIPP Model 

To deal more effectively with the evaluation of its numerous 
Title III projects, USOE has recently utilized the CIPP evaluation 
model authorized by Dan Stufflebeam (1967). This modsl orients 
data-gathering to decision-making, and vice versa. With appealing 
logic, Stufflebeam contends that the primary evaluation information 
gathered by the local staff should be that which is useful in making 
rational decisions about the local operation. This information is 
expected to be useful, when accumulated with that from other pro- 
jects, in making decisions at the system, state and federal levels. 

The tempo of this information flow is not to be dictated by the due- 
dates for formal reports, but should be cycled to conform to the 
decision-maker's needs for feedback information. 

The adoption of this model should be seen as a major advance 
by the USOE in its efforts to be of service to local activities. USOE 
has a legal and professional obligation to evaluate. Much of the think- 
ing of its staff concerning evaluation has been directed by Stufflebeam 
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to where it should be of great benefit — to the local level. The empha- 
sis on information for rational decision-making is consistent with the 
technologist's yearning for rationality. Unfortunately, the CIPP model 
seldom sates his appetite for cost determination and behavioral evi- 
dence. The principal defect of the CTPP model is that information 
gathered to aid local decision-making will seldom be the aggregate of 
information most useful for program-wide decisions. Additional in- 
formation is needed for federal and state decision-making and pro- 
gress-reporting. Evaluation activities beyond those operationalized 

by the CIPP model are required. 

When the chips are down— around USOE or in any bureaucracy 

the "old- reliable" way of evaluating a program continues to be: appoint 

a commission or advisory panel. If any large educational institution 
intensely desires to find out something, the formal sociological survey 
or experimental study is likely not to be relied upon.. Rather, a 
team of respected individuals is asked to visit, examine, and pass 
judgment. The fact that these commissions and panels often make 
thorough observations and draw insightful inferences does not excuse 
administrators from relying so heavily on such a primitive form of 
gathering and reporting information. Nor is it anything but an indict- 
ment of the educational-measurement specialists for not having 
demonstrated the superiority of more objective, reliable, valid, 
comprehensible and action- or iented methods of evaluation. 
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The Need for Information 

Each of the models discussed in the previous paragraphs has its 
strengths and weaknesses. No one, of them is a suitable model for all 
evaluation activity. A focus on student behaviors, or on management 
decisions, or on community needs, obligates the evaluator to ask 
special questions. Different foci require different bodies of informa- 
tion. What information to gather, what audience to serve, what 
language to use — the evaluator must make some tough decisions. His 

decisions depend on other decisions. 

The purpose of evaluation, Stufflebeam has told us, is decision- 
making. There is no justification for evaluation where there are no 
decisions to be made — and where there are decisions to be made, 
there is need for evaluation. There are many decisions to be made 
about the implementation of Title III, many different kinds of deci- 
sions. It is reasonable to expect that there will be many kinds of 

evaluation to guide those decisions. 

We can represent three major kinds of decisions and evaluation 

needs by three questions: 

What is the destination? 

Which path shall we take? 

What pace shall we set? 

This is a way of saying that we make decisions about our goals, our 
projects, and our tactics — many decisions about each. If our deci- 
sions are to be rational, overt, deliberate, as opposed to intuitive. 
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covert, and impulsive, we need information. Formal evaluation is 
a major source for that information. As different information is 
needed, different evaluation plans must be available. 

Of course, there is no one-and-only time at which we decide 
where we are going, which path to follow, or how to proceed. We 
make these decisions sometimes formally and with a bit of ceremony, 

[ usually informally, always repetitively, with conscious and uncon- 

[ 

I scious review, checks and balances. In addition to the three questions 

above, then, we should add these: 

Are we headed for the destination we chose? 

Are we on the path we chose? 

Are we proceeding in the manner we planned? 

These are status- checking questions, the questions of mechanical 

I governors, the questions of self- synchronizer units in the cybernetic 

system. They are important questions, leading to important deci- 

I sions — but they are different from the first three, and require differ- 
ent observations and information. 

i Evaluation Issues 

f “ 

r 

I 

What in'^'“ rmation for what decisions is indicated in the evalua- 

[ tion plan. Several Grand Plans, basic models, are available. Stuffle- 

\ 

beam presented one. Such people as Henry Walbesser (1965), Michael 
Scriven (1967), and I (1967), have presented others. These plans or 

\ 
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i models differ in the help they offer the decision-maker . I think it 
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would be helpful here to indicate some of the issues raised by the dif- 
ferent models. We evaluators of this colossus, Title 111, should be 
aware of these issues. 

The desire for the evaluation of educational programs is not 
new, nor is it a particularly strong desire among practitioners. But, 
the current abundance of models for evaluating seems new to me. 

Some new models emphasize the use of conventional tests while others 
do not. Other differences are (a) the importance of the classroom 
teacher as a developer of curricula, (b) reliance upon the developer’s 
intuitive, rather than rational, skills, (c) emphasis on subject-mat- 
ter-content goals, as opposed to intellectual-process-and- skill goals, 
(d) whether or not assessments will be limited to the developer's 
stated goals, and (e) the importance of building- in plans for reassess 
ing goals both during and after the developmental phase. 

One important inventory of evaluation issues was offered by 
Scriven. According to him, the important dichotomies are: 

1. Formative vs. summative evaluation, i. e. , evalua- 
tion during development to check the quality of com- 
ponents vs. evaluation after completion to check the 
quality of the whole. 

2. Professional vs. amateur evaluation: not only how 
well trained are the personnel in measurement and 
research methods, but also, are they professionally 
competent to deal with the subject matter, the teach- 
ing methods, the philosophical issues, etc. — and in 
addition, are the evaluators disinterested parties? 

3. Evaluation vs. process studies, i. e. , studies to 
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discover the worth of a program, against studies to 
discover the nature of a program, is the emphasis 
more on judgment or description? 

4. Intrinsic vs. payoff evaluation, i. e. , studies to dis- 
cover the quality of the input vs. studies to discover 
the quality of the outcomes. 

5. Comparative vs. noncomparative evaluation, i. e. , 
studies that compare alternate programs or proce- 
dures, with or without a so-called control group, 
vs. those which concentrate on the experimental 
program or procedure alone. 

6. Evaluation vs . explanatory studies — explanatory 
studies are expected not only to indicate the ”whys 
of the outcomes," as Tom Hastings (1966) puts it, 
but also to generalize beyond the specific curricula 
used. 

Title III Evaluation Components 

Title III of PL 89-10 calls for supplementary centers to facili- 
tate educational innovation in schools throughout the country. Like 
any federal program,—' it has its rationale, its purposes, its partici- 
pating projects, its procedures, its obstacles and its reward system. 
It also has a network of decisions, decisions which vary in kind and 
in purpose. An evaluation of Title III activities requires evaluation 
activites that vary in kind and in purpose as well. 

Let us consider the general components of program evaluation. 




—'Contrary to official usage, I am using the term "program" to 
indicate the entire Title III operation. The term "project" here refers 
to any local undertaking, simple or compound. 
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I cannot help bat oversimplify, bat let me isolate foar components: 



goals, projects, tactics, and oatcomes. First, goals . 



I will not sammarize the important emphasis Ralph Tyler 



(1950) and many colleagaes have given to goals, nor will I identify 



(as I did earlier) the goals of Title III. I want to observe here that 



goals are defined first in terms of wants , and only second in terms of 
words. Every attempt to translate wants into words will be less than 



perfect. Any list of goals (statements of behavioral objectives or 



otherwise) is saspect if it either appears to sapercede the wants or to 



represent them as anchanging. Any evalaation mast presame that 



stated objectives do not perfectly represent what is wanted. I have al- 



ready' made the point that goals are — and shoald be — flaid. I will 



make the point next that a valid list of edacational goals will contain 



competing and even contradictory goals. Goals are competitive in the 



sense that each parsait costs something. Unfortanately, the total of 



oar resoarces will always be less than the cost of parsaing all goals. 



We have to choose between oar goals. We assign priorities to them. 



We may do this conscioasly or anconscioasly. Bat we do it. It is a 



matter of choice, and we have no choice bat to choose. 



Goals will be contradictory. Often we seek incompatible oat- 



comes. We try to teach faith and skepticism. We try to instill deep 



appreciation, and yet provoke aspiration for something better. We 



try to give teachers an opportanity to be creative, yet we try to 
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bolster instruction through reasonable insistence on using the methods, 
topics, and materials of time-tested programs. We seek to serve a 
pluralistic society. Contradictory goals are to be expected in a plu- 
raiistic society. We cannot hope to pursue oniy goals that are per- 

fectly complimentary and universally wanted. 

Our evaluators must realize that goals are changing, competi 
tive, and even contradictory. A program evaluation is incomplete if 
it ignores this fact, if it goes no further than listing the specific goals 
at time zero. To understand the Title III operation and to ascertain 
its value, we are obligated to identify goals, ascertain priorities, re- 
veal the dynamics of changing priorities, and provide information for 
decisions about new goals and priorities. This is not to say that these 
things must happen first before we do anything else; nor is it to say 
that we must be as specific as a blueprint; but, as part of the opera 
tion, we must obtain some communicable representation 
of the wants this legislation was designed to alleviate. 

Title III Goals 

and of the transformation of these wants into other 
wants over a period of time. For this moment, 
please accept my representation of Title III goals 
in the figure on the right, a reminder, perhaps, 
that the Title III dollar does not alleviate every- 
one's wants. 
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Second, Projects. There are many pathways to the goal- 
destination. Title III administrators must choose among paths, 
among projects. , More plans will be proposed than can be followed, 
some must be selected, some must be rejected. Each project has 
its goals. Some projects will concentrate on a single goal; most 

pursue a complex. 

The rationale of a project 
may be independent in spirit— -it 
can emphasize local problems, 
local resources and local remedies, 
but it cannot be unrelated to general 
goals. Each local program proceeds 
in certain determinable ways to 
facilitate, or obstruct, the pursuit of national goals. The success of 
Title III is not merely the aggregate of progress and setback in the 
pursuit of local program goals. There is a national purpose as rep- 
resented by interpretations of the legislation— and, however close the 
fit, it will not coincide perfectly with the aggregate of project purposes. 

And Tactics. Even when the target and path are set, there are 
many modes of travel. No one tatic is necessarily right. Whether the 
Title III project is a multi-school dreadnaught or a lone knight- errant, 
each proposal cites special resources and techniques for doing its deed. 
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One generalization we have learned from educational practice and re- 
search is that almost any technique will work, at least in some cir- 
cumstances. We are often persuaded that a favorable combination of 
resources — community, school, and home— is more influential than 
the pedagogical techniques we employ. But we have no choice but 



to continue searching for better tactics 
to overcome the seemingly immutable, 
deleterious educational conditions in 
many communities. Our evaluation 
must consider the alternate tactics 
among which we search. 



1 
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Project Tactics 



Last, Outcomes . Any one project can be measured in terms of 
its outcomes. The changes will be many: student achievement and at- 

titudes, teacher maturation and alienation, organizational efficiency 
and compassion, renaissance of hope and initiative. There will be 
costs which, in a sense, are outcomes: expenditures of money, time, 

and effort; drains upon public and family confidence; etc. It is cer- 
tain that few changes will be easy to measure, and fewer still directly 
ascribable to the Title 111 effort. But there they are: costs and bene- 
fits, seen in different coin by different groups, various views of which 
are essential as description of the project, and essential as an infor- 
mation base for decision-makers. 

There is little sense in viewing the success of a project as 
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unidimensional. There is no single criterion outcome that is para- 
mount nor any composite that is a satisfactory 
index of success. As there are multiple goals, 
there are multiple successes and failures 
with any endeavor. A project is unfairly 
considered if its record is reduced to a 
single value-dimension. So it is with 

the overall Title III program. 

We seldom need such reduction. Sometimes a decision-maker 

must make an all-or-nothing decision, such as continuance or aband- 
onment. Then, some single continuum of value is an important fact of 
life. But most decisions pertain to degree of emphasis, here and there 
There are occasions when only one of several projects (or tac- 
tics) can be supported, where the goal priorities are similar, and 
where each promises a different profile of outcomes, or where each 
has registered a different profile of gains, with no profile clearly 
superior. When only one can be supported, a single continuum is again 
a necessary fact of life. The decision-maker must weigh outcomes in 
order to arrive at a single index of desirability. Clearly, one project 
must be rated superior to the others. But these are not common oc- 
casions. More often, the proposals competing for immediate funding 
are either not so similar in goal priorities, or else the superiority of 

one of the proposals is clear. 
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To make a claim or two, I will arrange the four components to- 



gether in Figure No. 2. The arrows indicate the existence of a cycli- 
cal influence, one on the other. Each component is influenced by local 



community conditions. The claims are. these: 



1 . 



A rational setting of goal priorities and funding of 
projects rests upon knowledge of: 



a. 

b. 



c. 



d. 



what goal alternatives (wants) there are 
what resources are available 
what types of projects can be funded 
approximate cost/benefit ratios for various 
types of projects 



2 . 



Certain projects are selected for funding. By care- 
ful plan, or merely in effect, a certain combination 
of goals is operationalized. 



Different tactics are selected for each project. The 
choosing of tactics continues through the duration of 
the project. By plan, or in effect, these tactics 
operationalize a certain combination of project goals, 
and through aggregation, they operationalize a cer- 
tain combination of program goals. 



A rational choice of tactics rests on knowledge of: 



a. 

b. 

c. 

d. 

e. 



project goals 

what tactics are available 

contingencies between tactics and outcomes 
what resources are available 
the costs of various tactics 



5. Outcomes are functionally dependent on tactics, 
given a set of local conditions. 



Goals can be defined as some combination of outcomes, 



The principal consideration of these comments is that a sequence of 
decisions is there to be examined both at the federal and local levels, 
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Figure No. 2. A representation of the major components of the Title 

III program. 
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A Decision - Evaluation Matrix 

To elaborate on the role I think evaluation should play in Title 
III, I have drawn up a matrix in Figure No. 3. The matrix shows the 
need for data for decisions. It does not show when it is gathered, 
nor — as the CIPP model does — how if flows back into the program 
operations. 

I set federal and local decisions /evaluations apart, although 
much of the activity at those two scenes is parallel. This is not to 



say that the aggregate of information gathered for local decision-mak 
ing will suffice for national decision-making, but it is clearly a. large 
part of what is needed. 

In addition to the six questions raised in the opening pages of 
this paper, I have raised several others concerning expected and ob- 
served outcomes. Included in the observed outcomes are the "hard 
data" for which there is such a rowdy official appetite these days. 
Those outcome data are important, but no more so than the other 
evaluation data indicated in the second and fifth columns of the mat- 
rix. At least that is the way I see it. 

In the third and sixth columns of the matrix, I have named 
eight different evaluation activities. I have borrowed from Scriven 
here, and" from my countenance paper (1967), but I have put a new 
slant on some of the terms. I would like to give them formal defini- 



tions, as follows: 
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1. Priority Setting — A study of wants under a given ra- 
tionale or philosophy, leading to preferential rating 
of goals, with implication for implementation. 



2. Feasibility Study— An estimation of the costs of over- 
coming various obstacles to implementing a given 
program or project. 



3. Environmental Survey — A gathering of information 
about the setting in which the program or project will 
occur, including its resources, social institutions, 
existing programs, personnel, organization, etc. 

4. Goal-Congruence Study — A study of the relatedness 
of goals of different undertakings, or of the related- 
ness of stated goals to those implied by practices. 



5. Intrinsic Evaluation — An analysis of the logic of the 
plans and activities of a program or project, provid- 
ing judgments of relevance and value of various com 
ponents. 



6. Payoff Evaluation — An empirical study of the degree 
to which observed outcomes approximate intended 
outcomes. 



7. Formative Evaluation— The empirical study of the ef- 
fects of various tactics, emphasizing functional rela- 
tionships potentially useful to other program develop- 
ment. 

8. Summative Evaluation — The empirical study of the ef- 
fects of a whole project under given environmental 
conditions, preferably with comparisons to alternate 
projects. 



These are the types of studies needed for even the first evalua 



tion responsibility — that of telling what the program has done, and how 
well the people like it. The methods for some of these studies are 
available; other methods are still to be developed. The fact that the 
nation's educators are not tooled-up to evaluate their programs is one 
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reason 



why we have not effectively evaluated Title lU. But. the blame 
for not being tooled-up is partly the fault of the U. S. Office of Educa- 
tion. The Commissioner has been just as willing as the American 
educator to make the short-range investment in teaching without a 
companion long-range investment in evaluation. 
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CONCLUSIONS AND RECOMMENDATIONS 



*/ 



by Richard I. Miller*” 



Is PACE really a "catalyst for change, " or is it becoming ser- 



vice money for local schools to do what they should be doing anyway? 



We hinted at the question in the First National Study of PACE, but now 



it needs to be faced head-on. Two basic changes in the nature of ESEA 



Title III — the turnover of control to the states, and the turnover of 15 



percent to the handicapped — force careful consideration of this ques- 



tion. 



CONCLUSIONS 



Before turning to some "final" conclusions, let us look at two 



other statements of conclusions about ESEA Title III. The first one 



*/ 



Identified in initial section. 



The conclusions and recommendations made in this section 
are those of the Director of the Study and do not necessarily reflect 
the unanimous views of the members of the study team. One excep- 
tion, however, is Report No. 2 which was unanimously agreed upon 
by all members of the study team. 
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was made in the January, 1967, report of the First National Study 
Team on PACE. It stated: 

"Considering everything— weaknesses and strengths, 
blunders and triumphs, politics and purity — Title III 
has thus far achieved outstanding success, probably 
more so than any other ESEA Title. Success is due: 

1. To its stimulating and fresh nature, which catches 
the imagination and zeal of the most dynamic and 
creative individuals in the public schools. 

2. It provides public school people with a unique op- 



local levels, and the bet is "paying off" contrary 
to some predictions that localism in education 
personifies status quo conservatism. 

3. Its success is due, in no small measure, to Office 
of Education Title III leadership, to that offered by 
some states, and to leadership offered at the pro- 
ject level. " 

The second statement was made in March, 1968, and received 
by unanimous endorsement of members of the Second National Study 
Team. The evaluative dimension of the report reads: 



"In the course of its work, this study team has examined 

several hundred Title III proposals and inspected close i 

to 200 projects in the field. Taken as a whole, considering 
the 2, 500 projects that have been funded over a period of 
two years, we believe that PACE is serving in many com- 
munities across the nation as a dynamic and positive force 

for educational improvement. | 

The study team feels that education has much at stake in 

the continuation of Title Ill's spirit of venture capital — j 

the first 'thinking money' many school districts ever | 

had — and in the success of the states in building upon this j 

thrust. Otherwise, if Title III should someday lose or | 

forget this major premise and early promise, it is |. 



portunity. In essence, PACE is betting on the 
vitality and vigor of American education at the 
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predictable that of necessity another fund will emerge 
elsewhere, quite possibly from those agencies dealing 
with the agony of cities, to recover and resume the 
unique quest that was Title Ill's. The nation has a 
right to expect that education will lead in its own re- 
newal. Title III is the sharpest tool to that end." 

How relevant are these two statements today — in November, 



1968? 

An answer is not simple, and any discussion of the matter 

should consider the various levels. 

First, at the federal level ; Close observers of the Washington 
scene over the past ten years, or since the NDEA legislation was 
passed, have noticed the increasing extent to which political considera- 
tions are determining educational directions. While the nature of our 
government rests upon the art and gamesmanship of politics, and we 
accept this, the education of a people must look elsewhere for its 
anchorage. Many of us are alarmed at the extent to which a handful 
of politicians and/or a cluster of organizations and special interest 
groups can manage educational policy. 

The two prominent examples of this danger are (1) the turn- 
over of ESEA to the states and, (2) the categorization of 15 percent 
of PACE monies for the handicapped. In both cases, overwhelming 
opposition among educators and PACE project directors would have 
been registered if anyone would have taken time to ask those most 
directly concerned. Just two examples of evidence available on 
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these points: In Report No. 5 on "The Views of 920 Project Direc- 
tors," one finds, even considering the tendency to "live with" the 
decision, strong and clearly dominant sentiment against the turnover 
to the states. This view also was found among members of the Second 
National Study Team. 

The 15 percent categorization has raised even greater opposi- 
tion. For example, the 230 conferees attending the President’s Na- 
tional Advisory Council Conference on Innovation in October, 1968, 
made 138 recommendations, as reported in the October issue of 
PACEreport . The single most frequently made recommendation 
called for elimination of the 15 percent for the handicapped. Again, 
the overwhelming sentiment of the members of the Second National 
Study Team opposed the 15 percent for the handicapped. 

A third consideration at the federal level concerns the chang- 
ing nature of the USOE staff, particularly those with primary respon- 
sibility for the PACE program. The attitude and spirit of the PACE 
staff have changed significantly over the past two years. The early 
enthusiasm and high expectations for ESEA Title III are all but gone, 

and so have some of the officials. Good officials do remain, and 

/ 

their dedication remains, but their willingness to "swing" and to "in- 
novate" has been diminished dramatically. (This change, however, 
cannot be attributedonly to changes in ESEA Title III; it comes also 



from the general wait-and-see attitude that is part of the transition 
to the Nixon administration. ) 

The problems of self-renewal that have been described very 
well by John W. Gardner are very evident when one observes what 
has happened with PACE personnel. Perhaps some mechanism or 
some procedure can be found that will provide the freshness, 
dynamism, dedication, and intelligence that are critical to the renew- 
al process. We need to know much more about the process itself. 

An important factor of ESEA Title Ill’s future success or failure 
may be the President’s National Advisory Council on Supplementary 
Centers and Services (NAC). As one of three presidentially appointed 
educational advisory councils, the NAC is charged with these functions: 

1. Review the administration of the general regulations 
for the Title; 

2. Review the operation of the Title, including its effec- 
tiveness in meeting the purposes for which the federal 
funds may be used; 

3. Review, evaluate, and transmit to the Congress and 
the President the reports submitted by state advisory 
councils to the National Council through their respec- 
tive educational agencies; 

4. Evaluate programs and projects carried out under the 
Title, and disseminate nationally the results of this 
evaluation; 

5. Make recommendations for the improvement of the 
Title and its administration and operation; 

6. Make an annual report of its findings and recommen- 
dations (including recommendations for changes in 
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the law) to the President and the Congress not 
later than January 20 of each year. The Presi- 
dent is requested to transmit to the Congress any 
comments or recommendations he may have with 
respect to the report. 

Speaking at the President’s Advisory Council Conference on 
Innovation about the NAC, its chairman, James Hazlett, chairman 
of the NAC (and Superintendent of Schools, Kansas City, Missouri), 
said that the Council "was established as a completely independent 
advisory body to the President and the Congress. The Council’s 
purpose is to review and evaluate on a national scale the impact of 
Title III in seeking imaginative solutions to educational problems; its 
mission also involves working closely with all groups concerned with 
PACE — State educational agencies. State advisory councils, the U. S. 
Office of Education — in order to promote better working relationships 
among these groups and to obtain essential feedback from all levels 
of operation. " 

Turning to the state level : The first national study found a dis- 

couraging picture at the state level. At the time — October, 1966, or 
one year after the Act's passage — only 10 states had full-time coordi- 
nators; and of the 37 states with part-time coordinators, nine gave 
10 percent of their time to it. In that same report, it was recom- 
mended that state departments "should receive a 4-percent allocation 
of the overall Title III appropriations for (1) development, stimula- 
tion, and for (2) administration of the title. " With the funding . 
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tiirnov6r to the sta.t©s 3.S W6ll sis th© p©rc©nt s©t 3.sid© in ©3.ch 

sta.t©'s appropriation for administration, on© can assum© that th© ©m- 
ploym©nt of ad©quat© stat© d©partm©nt p©rsonn©l to administ©r th© 

program is b©ing accomplish©d. 

But th© mor© xundam©ntal qu©stion is wh©th©r th© stat© d©part- 
m©nts — consid©r ing th©ir primary functions and traditional rol©s 
will b© abl© to k©©p aliv©, and ©nrich, th© spirit of cr©ativity and in- 
novation that has charact©riz©d th© PACE program. 

At th© pr©s©nt tim©, and continuing for at least several years 
into the future, a shifting of power and control from Washington to the 
various state capitals can be expected. .This flow is particularly evi- 
dent in education. (A flow within states, interestingly enough, is in 
the opposite direction — toward centralization of power and control in 
the state department of education.) 

State departments perform three vital functions: setting mini- 

mum standards, judgingthese standards, and serving as the educational 
fiscal agent for the state government. (The latter function alone is a 
major one when one considers that one-half or more of most state bud- 
gets are spent on education.) These important functions are regulatory 
and maintenance functions, and these are quite different from the inno- 
vative and creative and demonstrative type of programs that are needed 
for PACE if the original intent is to be preserved. 

As another input: on the survey of 920 PACE project directors 
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(Report No. 5), they were asked to list the greatest advantages and 
the greatest weaknesses of the turnover to the states. For advantages , 
they listed: 



1. 


Direct lines of communications 


51% 


2. 


Better knowledge of local problems 


24 


3. 


No advantage 


16 


4. More economical use of funds 

For weaknesses, they listed: 


9 


1. 


Politics 


61% 


2. 


Lack of leadership 


15 


3. 


Lower standards 
% 


12 


4. 


No weakness 


4 


Clearly, PACE directors are concerned that "politics" may 
come the final arbitrator of what is good education. 


In 


some states, the trend toward regionalization 


is another 



dication of the validity of their concerns. If a sta|:e is divided into 
geographical regions and each one is given "x" number of dollars for 
what they must call innovation and creativity, they have killed the 
spirit of PACE in their state, for all practical purposes. Whether they 
realize it or not, this approach turns ESEA Title III into another ser- 
vice program rather than one that provides "risk" capital. Service 

\ * • 

programs such as ESEA Title I and ESEA Title II are important and 



I 






















mm 



mmmmm 




S 



! 

I 






-242- 

can be very useful to school improvement, but they should not be con- 
fused with the spirit and intent of Title 111. 

A state-administered PACE program featuring creativity and 
innovation is being accomplished in some states, and some regionaliza- 




tions are promoting, the: intentions of PACE. But one cannot gen- 
eralize from the exceptions, and the state-level outlook for PACE, 

on the whole , is not promising. 

An important determinant of success or failure would appear 

to rest with the newly organized state advisory councils that have been 

mandated for each state. Speaking about the role of the state advisory 

council at the recent President’s Advisory Council Conference on In- 

* 

novation, Terrel H. Bell, Utah State Superintendent of Public Instruc- 



tion, said: 

It seems to me that advisory councils will be what 
make of them. Title III advisory councils will func 
tion on a high level if we appoint capable people, provide 
adequate and effective staff support, and place consider- 
able weight upon the advice of the council offers. To do 
otherwise is to be insincere in calling upon the valuable 
time of busy people who accept such assignments. . . . 



Roles of staff, advisory councils. Chief State 
School Officers, and state boards of education need to 
be clearly defined. Such deliberations will place heavy 
emphasis upon the processes of deliberation, weighing 
•'of priorities, and rendering of advice concerning deci- 
sions to be made. This is a key role in the administra- 
tion of Title III which can be placed by a representative, 
capable, and dedicated council. With such assistance. 
Title III programs in the ESEA will be more productive. 




iHiili 













■UilWMIJIP 







I 






- 243 - 



To this end, the function of the advisory council should 
not be considered as ’only advisory' but as ‘vitally and 
indispensably advisory.' 

Turning , finally , to the local level : The strength of PACE 
continues at the local level, with probably some diminution of enthur 
siasm and high expectations balanced with greater wisdom about pro- 
ject management and knowledge about how change takes place. Also, 
as many operational projects come to the end of their three -year 
cycle, professional security and project termination will have its toll 
upon the final six months of operation. 

The First National Study Report expressed this view about 

project directors: 

Within the field of professional education many dyna- 
mic, intelligent, creative, ambitious, and restless indivi^ 
duals can be found. They exist in every school system a- 
cross the Nation, and they can be a vital force in educa- 
tional improvement. Too many pf this group leave educa- 
tion because of low salaries and poor working conditions, 
to be sure, but probably more leave because of frustration 
and lack of challenge. 

PACE has become the natural home for this group. 

The special consultants and the directors have been im- 
pressed with the enthusiasm and intelligence found among 
the project directors. 

Evidence and observation subsequent to this quotation have not 
altered the nature or tone of what was said. 

Yet insufficient consideration has been given to how the PACE 

expertise might be applied to other educational problems. Consultants 

( 

still are drawn largely from the university and college circles, and 
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special programs in the change process have not been developed to 
harness PACE expertise. 

Considering the federal-state-local levels as well as other fac- 
tors that have been mentioned, what can be said about the state of 
ESEA Title III? On the whole, PACE remains healthy, although some 
signs of pre-mature old age are evident. The patient will grow more 
robust or continue downhill, at an accelerating rate, depending upon 
factors that are external to the patient. These factors, however, are 
not beyond his control. Project directors banded together in any one 
state may be able to assert effective pressure. 

RECOMMENDATIONS 







. 





’ 



There is life remaining in PACE, and ways need to be sought 
to enrich what is taking place. To this end, a series of recommenda- 
tions have been made in the various reports as well as in this one. 
Since each; report is .written to stand "on its own" as well as to serve 
the overall study, there is a slight overlap on some recommendations. 

Let us now turn to the various recommendations: 

Report No. li " Evaluation and PACE" : 






I. THERE SHOULD BE ESTABLISHED: 



A. A NATIONAL LABORATORY FOR THE STUDY OF 



EVALUATION 



B. A NATIONAL INFORMATION CENTER FOR EVALUA- 



TION 
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c. A NATIONAL GRADUATE SCHOOL FOR EDU- 

CATIONAL EVALUATION (This recommendation 
by Egon Gviba) 

II. EVALUATION COMPETENCE FOR ESEA TITLE III 
SHOULD BE DEVELOPED THROUGH SUMMER IN - 
STITUTES , IN - SERVICE EDUCATIONAL TELEVISION 
FEATURING VIDEOTAPES DEVELOPED FOR NATIONAL 
USAGE . AND SPECIAL MANUALS . 

NDEA: institutes might well be developed on PACE evalua- 
tion. Twelve, six week summer programs might make a significant 
dent in the PACE evaluation, especially if this cadre would be on call 
to help other PACE projects. 

III. EVERY PROPOSAL SHOULD AMPLY DEMONSTRATE 
THAT OBJECTIVES HAVE BEEN CONSIDERED AT THE 
GENERAL AND SPECIFIC LEVELS. 

Furthermore, learning or behavioral objectives should.be 
related to program activities, and the types of evaluation used should 
be related to activities. 

Again, the proposals reflect a serious weakness in 
American education; namely, giving lip-service to objectives. The 
tendency is to develop an idea in terms of bringing about some im- 
provements, but rarely do project developers force themselves in the 
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difficult position of making precise decisions about objectives. But 
this initial, step is essential in order for effective evaluation. 

IV . PROCEDURES FOR EVALUATION SHOULD CLOSELY 
REFLECT THE NATURE OF THE TASK OR PROJECT 
TO BE EVALUATED . 

The current interest in cost effectiveness and cost ben-e- 
fit studies has prompted greater attention to hard data, and this atten- 
tion on the whole is desirable, but PACE directors must not try to 
force hard data procedures upon unlikely situations. Robert Havig- 
hurst, in commenting on the problems of evaluating Supplementary 
Educational Centers, points out that ’’the programs of the Centers 
tend to be broad, and rather vaguely defined. They usually propose 
to create new courses of instruction with new teaching materials, 
or to train teachers and counselors for new roles. They do not lend 
themselves to an experimental design, with experimental and control 
groups of students and statistical tests of various hypotheses. 

V. EVERY PACE PROPOSAL SHOULD BE REQUIRED TO 
HAVE A SEPARATE BUDGET ITEM FOR EVALUATION , 
AND THE AMOUNT OF THIS FIGURE SHOULD NOT BE 

less than five percent of the total budget . 

V eiry little or ho budgetary commitment to evaluation r e 
suits in very little or nothing. The return expected is directly related 
to the investment made. Proposals simply must have a well-defined 
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and adequate evaluative expenditure to expect sound results. The five 
percent figure is not based upon research but upon experience with a 
few proposals that seem to have an adequate evaluation scheme. Some 
evaluation schemes go up to ten percent of the total budget. 

VI. NEW GUIDELINES NEED TO BE DEVELOPED BY THE 
USOE THAT WILL STRENGTHEN PROJECT ASSESS- 
MENT. 

The problem of guidelines is complicated by the widely 
varying differences among PACE projects, ranging from a single task, 
single school project with modest financing to multi-task, multi-dis- 
trict effort with several hundred thousand dollar expenditure. The 
objectives established for these two projects would be vastly different 
and therefore so should be the evaluative procedures. 

; The problem of guidelines is further complicated, by dif- 
ferences in types of activities. Elliot Eisner, for example, points out 
that "it should be made clear at the outset that the evaluation of learn- 
ing in the cultural arts is a task beset with a variety of special diffi- 
culties. . . . The context in which evaluations in the cultural arts reside 
is one that tends to have little disposition toward objective evaluation 
and few instruments appropriate for evaluation when unique objectives 
are formulated. " 

VII. THE USOE SHOULD DEVELOP SIMPLE YET 



ACADElvlICALLY SOUND MATERIALS ON THE 
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THEORY AND PRACTICE OF EVALUATION THAT WILL 
PROVIDE CONCRETE ASSISTANCE TO PROJECT DIREC - 
TORS . 

Financing of the ESEA Title III largely will be turned over 
to the states within two years, yet the Commissioner will maintain the 
responsibility for approving all state plans, and within these could be 
required detailed evidence that evaluation was being given serious at- 
tention within the state. The availability of sound materials would 
have an important role in improvement of projects within the various 
states. 

Report No. 4; " Analysis and Evaluation of 137 ESEA Title III 

Planning and Operational Grants ' ' 

I. EVERY PROJECT PROPOSAL SHOULD BE REQUIRED 
TO SUBMIT EVIDENCE THAT THOSE DEVELOPING 
THE PROJECT HAVE A GOOD GRASP OF THE LOCAL 
AREA . BOTH IN TERMS OF NEEDS AND RESOURCES . 

This evidence does not have to be a needs assessment 
study if the project is a single idea or a program; still, some evi- 
dence of local considerations should be evident. 

II. EVERY PROPOSED SUPPLEMENTARY CENTER TYPE 
OF PACE PROGRAM SHOULD INCLUDE A REASONABLY 



THOROUGH NEEDS ASSESSMENT STUDY. 
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' A statetnent or assessrtierit of needs should include the 
identification of goals, processes for goal attainment, and specifica- 
tion of areas of greatest needs and deficiencies. Most terminal re- 
ports included little or nothing about how issues or programs were 
selected, and, in many other instances, this essential aspect of the 
report covering the supplementary center type of project was either 
minimized or omitted. 

III. STATE DEPARTMENTS OF EDUCATION NEED TO GIVE 
CAREFUL CONSIDERATION TO THE TYPE OF TERMI - 
NAL REPORTS THAT WILL PROVIDE A FITTING 
CLIMAX TO A PACE PROJECT . WILL MEET LEGAL 
REQUIREMENTS OF REPORTING . AND WILL ALLOW 
ESSENTIAL FINDINGS TO BK DISSEMINATED EF - 
FECTIVELY . 

Our s^tudy found that most projects omitted one or more 
types of information, such as: project title, type of project, grant 

number, period of time, amount of the grant, number of students to 
be served, cost per student, number of school districts involved, the 

name of the state, and so forth. 

The study team had no idea how sloppy, inaccurate; ahd 

incomplete it would find the final reports, in most cases . Those who 
submit such reports are guilty of professional negligence and fiscal ir- 
responsibility, and they need to be dealt with accordingly. If this 
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message seems overstated, one needs only to examine the end of pro- 
ject reports submitted by many projects. 

IV. INVOLVEMENT OF COMMUNITY RESOURCES AND 
PERSONNEL SHOULD BE MORE CAREFULLY CON- 
SIDERED ; IT SHOULD BE REALISTIC AND S HOULD 
HAVE ADEQUATE FOLLOW - THROUGH. 

No promises should be made that cantiotbe kept; no ob- 
ligations should be incurred that cannot be met. 

A majority of the projects studied are guilty bf overex- 
tension and superficiality on community relations. The project de- 
velopers promise too much, involve too many, and analyze too little 
the HOW of effective community involvement. 

V. ALL PROJECTS SHOULD HAVE EFFECTIVE EVALUA - 
TION PROCEDURES — EFFECTIVE IN TERMS OF ' 
STATED OBJECTIVES AND PLANNED PROGRAMS . 

The call for better evaluation is an old saw, if the three 

years of PACE history is told; but the call needs to be made again and 
again. 

In only one or two instances out of 94 planning projects 
can one glean from the proposal a serious and sophisticated concern 
about evaluation— a concern that viewed evaluation as a vital part of 
the day-to-day monitoring process as well as a judgmental decision 
reflecting success or failure of the program. 
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VI. EVERY PACE PROPOSAL SHOULD HAVE A SEPARATE 
BUDGET ITEM FOR EVALUATION. AND THIS FIGURE 
SHOULD NOT BE LESS THAN FIVE PERCENT OF THE 
TOTAL BUDGET. 

^ Only a small number bf the terminated projects included 
plans for evaluation, and even these appeared to be afterthoughts or 
were non-integral parts of the project structure. While there is rea- 
son to believe that evaluation has improved during the last year, the 
level is still far below what is desirable and what is needed. 

Vn. MORE EVIDENCE OF PLANNING SHOULD BE REQUIRED 
IN FUTURE PACE PROPOSALS . 

The study team was aware Of the dilemma between over- 
planning and over- structuring on the one hand, and a relaxed, prag- 
matic approach to design on the other; but evidence gained from the 
137 terminal reports lead us to believe that greater emphasis should 
be placed on planning and design in all future proposals. 

Vm. PROVISIONS FOR CONTINUATION AFTER TERMINA - 
TION OF ESEA TITLE U1 FUNDING SHOULD BECOME 
MORE EVIDENT IN THE FUTURE . 

The newness of PACE, the unexplored, perimeters of its 
guidelines, and the unknown labyrinths of federal assistance have all 
mitigated against serious consideration of what might take place when 
the planning grant ended. But as we look ahead, profiting from the 
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past, continuation considerations should become more important with- 
out becoming a requirement for approval. 

IX. FUTURE PACE PLANNING GRANTS SHOULD BE AL- 
LOCATED ON A SHARING BASIS WITH LOCAL COM - 
MUNITIES — SOMETHING IN THE DOLLAR RANGE OF 
8 OR 10 TO ONE . 

The study team found that where local funds were com- 
mitted to the project, it was better planned, the objectives more 
clearly stated, and the procedures for realizing the major goals of 
the project were more carefully developed. 

Report No. 5: " The Views of 920 PACE Project Director s" 

I. MEETING OBJECTIVES . NEEDS OF THE AREA , INNO- 
VATIVENESS AND CREATIVITY . AND MERITS OF THE 
PROPOSAL SHOULD BE GIVEN PRIMARY EMPHASIS 
IN DEVELOPING AND EVALUATING PACE PROJECTS. 



PACE stands for Projects to Advance Cr eativity in Edu- 
cation. This obvious point is easy to forget when a project director 
states that his project is: 



serving a. sizeable number, 
providing good public relations, 
producing results, 

assisting the regular school program, 
and. so. forth. 



The descriptors are legitimate and may be important, but they are not 
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the focus of PACE. It is very important that the innovative and crea- 
tive ideals of PACE be carefully guarded. American education is in 
constant and considerable need of cutting edge, diverse approaches to 
common and uncommon problems. 

U. STATE ADVISORY COUNCILS SHOULD BECOME POWER - 
FUL INSTRUMENTS . THEMSELVES ERRING ON THE 
SIDE OF CREATIVITY AND DYNAMISM RATHER THAN 
PASSIVITY AND APPROVAL . 

. This volume has not dfealt with state advisory councils 
but it has been concerned with the primary thrusts of the PACE pro- 
gram, and since state advisory councils — a new mechanism — are 
importantly situated with respect to these thrusts, it would seem ap- 
propriate to bring in the state advisory councils at this point. 

. At this early juncture it is impossible to judge the quality 
of the state advisory councils, but as Terrel H. Bell, Utah State 
Superintendent of Public Instruction, points out: "It seems to me that 

advisory councils will be what we make of them. Title III advisory 
councils will function on a high level if we appoint capable people, pro- 
vide adequate and effective staff support, .and place considerable 

a/ 

weight upon the advice the council offers. " 

d. / 

“ Terrel H. Bell, "The State Advisory Council, " Conference 
on Innovation . (Report by the President's National Advisory Council 
on Supplementary Centers and Services, November, 1968), p. 44. 
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m. STATE ADVISORY COUNCILS MUST TAKE EVERY CAU- 
TION AGAINST UNDESIRABLE POLITICAL INTERESTS . 
WHICH CAN INCLUDE GEOGRAPHICAL CONSIDERA - 
TIONS AND PATRONAGE . 

The problem of excessive politicalinterests in some 
states may be an excessive albatross for organized innovation. At 
this early stage, one can cite a few instances where the dynamic and 
exciting edge of PACE has been compromised by political interests. 
While politics is a vital part of our way of life, our children and youth 
are the losers when selfish political interests of a few take precedence 
over educational interests of many. 

Perhaps an open awar eness of thb danger s of excessive 
political considerations is the best safeguard against it, along with 
carefully designed procedures for project development, evaluation, 
and dissemination. 

As more political and educational power shifts from the 
federal and the local levels to the state level — a trend that is now 
several years old and likely to continue for many more years — many 
observers believe that new approaches and programs will be required 
at the state level if the challenges of new opportunities are to be met. 
In his January, 1967, .. Inaugural Address, Washington's Governor 
Daniel Evans said: "State governments are unquestionably on trial 

today. If we are not willing to pay the price, if we cannot change 
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where change is required, then we have only one recourse. And that 

is to prepare for an orderly transfer of our remaining responsibilities 

a/ 

to the federal government. 




IV. WAYS OF CONTINUING SOME PACE PROJECTS BE - 
YOND THREE YEARS SHOULD BE FOUND . 

Sound investment of public monies for education requires 
that some, probably few, PACE projects should be continued beyond 
three years, but probably not more than five years in any case. We 
know now that three years is altogether too short a period of time for 
some projects and an excellent time span for many others — probably 
a majority of them. 

Many potential problems loom ahead if the three-year 
grant period is lengthened or made open-ended for all projects. It can 
mean that fewer new projects can be started; that the state will be 
saddled for more than three years with average or less-than-average 
projects; and that the tempo of individual projects may become less 
dynamic. 

. At this tinie, it would seem uriwise to have a general ex- 
tension beyond the three years, but it seems equally unwise to not have 
some sort of status that will allow an extension (a) for the exceptional 



/ 

““ Quoted in Committee for Economic Development, 
ing State Government . New York: The Committee, 1967, 
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p. 10. 
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projects and (b) for the exceptional project that requires a longer 
developmental period. 



V. SUBSTANTIALLY GREATER FUNDS SHOULD BE AP- 



PROPRIATED FOR ESEA TITLE III. 



Evidence obtained on the 1966 and 1 968 surveys indicates 
very clearly that project directors, those who should know, need 
more funds. 

The turnover to the states may be a hopeful sign for in- 
creased funds. Heretofore, the PACE program had no organized con- 
stituency; now it has 50 powerful ones, and perhaps some organiza- 
tional support also. An organized effort to increase the appropriate 
funding level is certainly in order. While the program is far from 
perfect, it is serving well the cutting edge dimension of American 
education. In other words, the increase in PACE funds is a good in- 
vestment of public money. 









Report No. : ’ ' PACE : Catalyst for Change ' ' 

I. THE 15 PERCENT CATEGORIZATION FOR THE HANDI - 
CAPPED SHOULD BE RESCINDED . 

There is no educationally sound justification for singling 
out the handicapped. One can make at least as good an educational 
case for singling out the gifted, and of course a much stronger case 
can be made for the gifted in terms of our national interest. 
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R eactiofi to the handicapped provision is clears; {!)' An 
overwhelming majority of educators and PACE project directors are 
opposed to it, and (2) the 15 percent has taken all new monies in 
many states, and the provision has forced some ongoing projects to 
fit in something for the handicapped, thereby violating the original 
spirit and intent of ESEA Title III. 

•This recommendationmust not be interpreted as anti- 
handicapped. Such an interpretation really misses the point. The 
handicapped category already is taken care of under provisions of 
other Titles. The recommendation opposes not the handicapped, 
per se . but the effort to break ESEA Title III into categories and 
thereby destroy not only the flexibility in the program but also the 
freedom of choice that has provided a milieu conducive to creativity 
and innovation. 

II. THOSE RELATED TO PACE SHOULD ACCENTUATE 
THE POSITIVE WITHOUT IGNORING THE NEGATIVE . 

In spite of the many problems and weaknesses that can 
be related to ESEA Title III, one needs to view the Title in somewhat 
the same way as the football coach with a good team. His next op- 
ponent, however, is the pride of the region and a three -touchdown 
favorite. Rather than wringing his hands and thinking "all is lost and 
may Sunday come as kindly as providence will allow, " he assesses 
the strengths and weaknesses of both teams as accurately as possible — 
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avoiding self-deception at all costs. His game plan will be based upon 



the positive belief that his team can win. After all, it is the only team 



he has — and miracles can be made to happen! 










